Configuring Workloads for Microarchitectural and Side Channel Security

ID 768538
Updated 2/13/2023
Version Latest
Public

author-image

By

Incidental channels in computing systems are unintended communication channels formed by valid properties such as execution time, power consumption, and the use of shared resources. When data flows through an incidental channel, both data values and metadata (for example, memory addresses being accessed) may be inferable by malicious actors. Malicious actors may potentially use the incidental channel as a covert channel or in a side channel attack to extract secret information, such as cryptographic keys. 

It’s important to recognize that incidental channels are an inevitable result of shared resources. Furthermore, incidental channels are important features of advanced hardware and software that provide customer value through improved user experience, performance, and resource utilization. 

Intel takes the threat of covert channels and side channel attacks seriously. Given the nature and benefits of incidental channels, Intel believes removing all incidental channels from computing systems is not in customer’s best interests and is not feasible, nor is it feasible to completely prevent the intentional misuse of incidental channels. 

Intel’s strategy for mitigating potential covert channels or side channel attacks that use hardware-based incidental channels involves both hardware and software: optimizing hardware to avoid direct exposure of data values via incidental channels; software following secure coding practices to address data values from being unintentionally revealed via metadata. 

Over the past decades, the domain of incidental channels has constantly evolved. Malicious actors have gone from learning cryptographic keys via physical power measurements or a program’s execution time to extracting information from microarchitectural components, such as by inferring memory access patterns from caches. More recently, incidental covert channels have become a tool for adversaries to extract information in the context of other microarchitectural features, such as transient execution, and in addition, the effects of physical incidental channels such as power consumption have turned out to be accessible by software. This increased impact and accessibility of incidental channels to software has made the collaboration between hardware and software more critical than ever before to effectively and efficiently address microarchitectural incidental channels. 

This document provides an overview of common incidental channels found in CPUs, possible security threats these incidental channels might pose, and strategies to mitigate the potential impact. Intel has released guidance on coding principles that software developers can follow when handling secrets to minimize the risk of incidental timing side channels.

Intel’s position is that hardware and software must work together to minimize the risk to workloads posed by incidental channels being used as side channels.

Background

Previous documentation describes channels in computing systems as a medium to transmit or signal information. That broad term can be further segmented and defined as follows: Legitimate channels are intentional and used to transmit information over the system interfaces, such as ethernet, shared memory, and IPC sockets. All other (unintended) channels in computing systems are considered to be incidental channels. These are formed by properties such as execution time, power consumption, and resource contention. It’s important to state that incidental channels are not inherently dangerous, but may pose a security threat if they are used by a malicious actor as part of a side channel attack or as a covert channel. In a side channel, an attacker controls the receiver and observes information that is unintentionally transmitted or leaked by a vulnerable application. In a covert channel, an attacker controls both the transmitter and the receiver of the channel, for example, in an effort to communicate information and bypass access control mechanisms. 

The risk of communicating secret data across security boundaries using covert channels was first described in the 1970s in the context of program confinement (Lampson, 1973). Likewise, the security implications of side channels were first identified in the 1990s by describing the relationship of RSA cryptographic keys and execution time (Kocher P. C., Timing attacks on implementations of Diffie-Hellman, RSA, DSS, and other systems, 1996). In this side channel attack, a vulnerable application computing RSA signatures unintentionally leaked information about the cryptographic key by its execution time, which allowed an attacker to fully recover the key. Since then, many other incidental channels have been found, such as power consumption (Kocher, Jaffe, & Jun, Differential power analysis, 1999) (Chari, Rao, & Rohatgi, 2002), electromagnetic (EM) emanation (Gandolfi, Mourtel, & Olivier, 2001) (Agrawal, Archambeault, Rao, & Rohatgi, 2002), sound (Genkin, Shamir, & Tromer, 2014) (Asonov & Agrawal, 2004), temperature (Brouchier, Kean, Marsh, & Naccache, 2009) (Hutter & Schmidt, 2013), and sensor data (Cai & Chen, 2011).  

While incidental channels can be used by malicious actors to pose a security risk, all incidental channels identified so far are unintended side effects of performing legitimate computations. For instance, the intrinsic power characteristics of semiconductors, which inherently depend on the data being operated on, can result in a power incidental channel. Natural software coding patterns as well as hardware and software implementations that improve user experience, performance, and resource utilization can cause timing incidental channels. For instance, the natural way of implementing control flow creates an incidental channel where the execution time depends on the condition variable. Similarly, hardware and software techniques such as data compression and caching have varying execution times, depending on the application’s data. 

A system’s defined threat model may require mitigating the potential security effects arising from incidental channels. As incidental channels can originate from many different places within an IT system, both hardware and software vendors should implement recommended measures for closing incidental channels, such as by partitioning shared resources or by implementing secure coding practices (Guidance for System Administrators to Mitigate Transient Execution Side Channels, Guidelines for Mitigating Timing Side Channels against Cryptographic Implementations, Security Best Practices for Side Channel Resistance). Sharing responsibility for security between hardware and software helps to accommodate different security and performance requirements among users, and helps maintain and promote good user experiences and effective resource utilization.

This document gives an overview of the primary types of incidental channels in CPUs, the potential security threat these incidental channels may pose when sought to be exploited by malicious actors, how Intel addresses incidental channels, and strategies to mitigate their impact. 

Incidental Channel Taxonomy

Incidental channels in CPUs can be labeled with multiple identifiers, including physical channels and software-accessible channels, as shown in Figure 1. Physical channels are the result of physical properties of integrated circuits (ICs) that change depending on the computations being done. For instance, the power consumed by CMOS gates and buses depends on their switching activity, which is determined by the data being processed on the IC. This dependence creates an incidental channel that could be used to form a covert or side channel. Other physical properties such as temperature and EM radiation likewise depend on system load and transistor switching activity, and hence create incidental channels. Physical channels usually require close physical proximity to a device to read information on the incidental channel, for example, by attaching an oscilloscope to the system or using an EM antenna.

Software-accessible channels generally can be used directly from within software and hence may provide a large attack surface.  Some physical channels can also be accessed by software; for example, the Running Average Power Limit (RAPL) interface allows software to tap physical channels too. Intel published guidance on mitigating side channel vulnerabilities using this incidental channel.

Figure 1: Example of taxonomy terminology of incidental channels on CPUs

Software-Accessible Incidental Channels

This document focuses on incidental channels in CPUs that are accessible from within software. Software-accessible incidental channels can be categorized in different ways, depending on the channel:

  • Results from data-operand dependent timing behavior or from data-operand independent timing behavior,
  • Is stateful or stateless,
  • Or if the channel is used in transient execution attacks or in side channel (non-transient execution) attacks.

Data Operand Dependent Timing Incidental Channels

Data operand dependent timing incidental channels result from data operand dependent behavior. A cause of data operand dependent timing channels is data operand dependent optimizations within the CPU microarchitecture, which may result in measurable differences. For instance, div instructions are complex, multi-cycle instructions, which may optimize the operation depending on its operands to avoid delaying execution for simple input values, such as division by 1, to the same latency as needed for more complex input values. 

Such conditions are not limited to CPU designs, but apply to IT systems generally. A very simple password check, for example, may be implemented by iterating over all characters of the correct and the provided password and testing each character for equality, as shown in Example 1.

int check_password(char *user_password, char* correct_password) { 
  int equal = 1;
  while (*user_password != 0 && *correct_password != 0) {
    equal &= (*user_password++ == *correct_password++);
  }
  equal &= (*user_password == *correct_password);
  return equal;
}

Example 1: Password check with varying execution time depending on the length of the correct password and the provided password

As this password check terminates, as soon as either the correct or the provided password ends, the execution time of this password check leaks information about the length of the password. If an optimized implementation of this check aborts early at the first mismatched character, as shown in Example 2, this optimized password check’s execution time leaks information about the correct password’s characters. Such an example shows how intuitively implementing a simple string comparison in software leads to a timing incidental channel that could potentially be used to leak secret information. 

int check_password_opt(char *user_password, char* correct_password) { 
  while (*user_password != 0 && *correct_password != 0) {
    if (*user_password++ != *correct_password++) {
      return 0;
  }
  return (*user_password != *correct_password);
}

Example 2: Password check with execution time depending on the number of leading characters being equal for correct and provided password

Data Operand Independent Timing Incidental Channels

Data operand independent timing incidental channels may reveal metadata, most commonly accessed code and data addresses and the pattern of these accesses, about the current execution rather than the actual data operands being processed1.  Note that depending on the software, this metadata can yet encode sensitive information. One root cause of data operand independent timing incidental channels is resource contention in system resources that are shared between different users. This contention affects execution time of the running program, and thus its execution time may leak information about the system resources it uses. Below are some examples of data operand independent timing incidental channels:

  • Sibling hyperthreads on a physical core contend for execution units. An adversary running on a sibling thread can repeatedly issue instructions that occupy specific execution units to create contention in these execution units. Measuring these instructions’ execution time could allow an attacker to gain insights into how often a victim application running on a sibling thread used a particular execution unit. (Aldaya, Brumley, ul Hassan, García , & Tuveri, 2019)
  • Physical cores on the same die compete for the memory bus. An adversary can create contention on the memory bus by issuing many memory requests. Measuring the memory accesses’ execution time may allow the adversary to learn about concurrent memory accesses by a victim application. (Wu, Xu, & Wang, 2012)
  • Physical cores on the same die compete for the ring bus. An adversary creating contention on the ring bus could learn about the victim requests’ destinations on the bus. (Paccagnella, Luo, & Fletcher, 2021)

Transient Execution Attacks

As highlighted in a prior article, transient execution attacks seek to access information about computations done during transient execution in out-of-order CPUs by transmitting the results of such executions over a covert channel. While computations occurring during transient execution may never become architecturally committed due to mispredictions, incidental channels may allow encoding transient results into microarchitectural states that can later be read out by an attacker. Hence, transient execution attacks combine a speculation mechanism with an incidental channel, as outlined in this document, such as stateful, stateless, data dependent timing, and data independent timing channels. The root cause of transient execution attacks are the underlying prediction and speculation mechanisms in the microarchitecture that improve instruction-level parallelism and thus performance; such attacks combine such mechanisms with incidental channels caused by shared resources. 

Different speculation primitives and disclosure gadgets can be used to access and transmit secret information during transient execution. Intel has published guidance for the many of these:

Rogue Data Cache Load (RDCL) (Rogue Data Cache Load, n.d.; Lipp, et al., 2018), L1 Terminal Fault (L1TF) (L1 Terminal Fault, n.d.; Van Bulck, et al., 2018) and Microarchitectural Data Sampling (MDS) (Microarchitectural Data Sampling, n.d.; Van Schaik, et al., 2019; Schwarz, et al., 2019; Canella, et al., 2019) use a faulting instruction as a speculation primitive.

Bounds Check Bypass (BCB) (Bounds Check Bypass, n.d.; Kocher, et al., 2019) and Branch Target Injection (BTI) (Branch Target Injection, n.d.; Kocher, et al., 2019) result from branch predictors. 

Speculative Store Bypass (SSB) (Speculative Store Bypass, n.d.) is caused by memory disambiguation prediction.

In all of these examples, covert channels are used to encode the results of transient execution.

Side Channel (Non-Transient Execution) Attacks

Side channel attacks that do not rely on mechanisms related to transient execution are often the result of data-dependent control or data flow inside applications. For instance, the memory access patterns in software may create contention in a cache that can be measured by an attacker, or the execution time of a cryptographic algorithm may depend on the bits of the cryptographic key, such as for a simple square-and-multiply implementation of RSA (Kocher P. C., Timing attacks on implementations of Diffie-Hellman, RSA, DSS, and other systems, 1996). 

Stateful Channels

Stateful channels are incidental channels that modify microarchitectural state, such that changes to this microarchitectural state are measurable at a later point in time. For instance, accessing an address loads a line into the cache and evicts another line from the same cache set. This change in cache state could be used by an attacker to infer information about the loaded address at a later point in time, meaning that the attacker does not necessarily run concurrently with a victim application. The microarchitectural state changes causing these incidental channels are commonly part of stateful optimizations that allow hardware to speed up software. 

Stateless Channels

Stateless channels are incidental channels that do not modify microarchitectural state and can only be used as a side channel or covert channel at a specific time. For example, contention on an execution port is only measurable while the contention is happening. As a result, an adversary needs to run concurrently to the victim application. Stateless channels are usually the result of contention and originate from sharing hardware resources concurrently between applications. 

Intel’s Approach to Incidental Channels

Incidental channels are ubiquitous in computing systems and occur in both hardware and software. These are often a result of desired product enhancements: higher performance, increased usability, better quality of service, more efficient resource utilization, and lower overall cost. Depending on the specific application’s security goals and its threat model, incidental channels in hardware and software may be acceptable or may require specific mitigations. For applications with security requirements, incidental channels have the potential to be used by malicious actors as both covert channels and side channels.

Covert Channels

Covert channels allow information to be transmitted across security boundaries, where an attacker controls both the input and output of the channel. As discussed above, there are many incidental channels on modern processors which potentially can be used for covert channels, including each level of the cache, translation lookaside buffers (TLBs), paging structure cache (such as page-directory entries (PDE) or extended page table (EPT) page-directory-pointer table entries (PDPTE)), microarchitectural predictors (for example, memory disambiguation predictor, branch predictors, etc.), prefetchers, and shared resources including buses and execution units). Many of these have duplicate versions (for example, separate TLBs for instruction-side and data-side) and replacement algorithms (for example, Least Recently Used (LRU)) that can be their own incidental channels. 

The threat model of covert channels requires attackers to be able to access relevant, secret information before exposing it via the covert channel. Mitigating covert channels may thus be desirable in certain limited environments, such as in multi-level security (MLS) systems, where an attacker has legitimate access to classified information and must be prevented from communicating it. However, since covert channels themselves are not typically the root cause of security issues, typical threat models should focus on mitigating root causes as suggested in the following paragraph.  

Covert channels are inevitable whenever resources are shared, and the performance overhead of mitigating all covert channels in a computing system can be unacceptably high. The many different types of covert channels make it extremely difficult to prevent attackers from communicating secret data via any of these covert channels once the attacker has read the data. Intel’s recommended approach to covert channels is ensuring that attackers do not have access to secret data in the first place, instead of trying to block specific covert channels that can be used to exfiltrate that data. Intel focuses on providing hardware and software support for mitigating vulnerabilities which could allow attackers unauthorized access to secret data. This approach implicitly helps prevent attackers from leaking secret data via covert channels. 

If system maintainers decide to try and prevent covert channels, strong isolation needs to be enforced by avoiding shared resources via physical isolation between applications that may potentially collude.

Side Channels

Side channels are a broader security concern, since they can leak secret information from a vulnerable application to an attacker, where the vulnerable application sends and the attacker receives secret data via an incidental channel. Since incidental channels can result from software and hardware design choices, efficiently mitigating side channels may require efforts in both hardware and software.

Side channels may directly leak data that is being processed but can also leak metadata that exposes information about executed code and accessed memory. Many of the incidental channels discussed earlier reveal metadata, most commonly accessed code and data addresses (including transient code/accesses2) and the pattern of these accesses. Software developers concerned about side channels should seek to ensure that the metadata, such as code being loaded and memory addresses being accessed, do not reveal secret data. Conversely, the role of hardware is to protect secret data being processed from being revealed by side channels, rather than protecting metadata. The type of data that is revealed and the level of attacker control on what is revealed is more critical than whether the attack vector is a side channel, transient execution, or even a non-side channel vector. 

Intel provides guidance to developers to help them understand and mitigate side channels, as well as hardware features to efficiently mitigate side channels working together with software. For example, Intel’s Security Best Practices for Side Channel Resistance guidance and Intel’s Guidelines for Mitigating Timing Side Channels Against Cryptographic Implementations give a high-level overview of how to efficiently avoid secrets from being revealed via side channels in a variety of different circumstances, such as by constant-time programming principles.

Protecting Against Side Channels in Software

There are several steps that developers are advised to take to protect software against side channels:

  1. Threat modeling: Specifying a threat model and learning about which secrets are at risk from which types of attacks helps system administrators and developers understand what security objectives they are seeking to meet and what countermeasures should be put in place.
  2. Secure coding: Following secure coding practices helps ensure that control flow, data flow, and memory access patterns are independent from the program inputs to protect against side channels in hardware and software. (Software Security Guidance Best Practices, Security Best Practices for Side Channel Resistance).  
  3. Process isolation: Isolating workloads in separate processes reduces the threat surface in the presence of transient execution vulnerabilities. (For more information, refer to Managed Runtime Speculative Execution Side Channel Mitigations).

System administrators should keep their system up to date by:

  1. Applying microcode updates: Intel regularly ships microcode updates for its supported processors that include mitigations against transient execution vulnerabilities. 
  2. Applying OS updates: OS vendors may include mitigations against newly discovered vulnerabilities in conjunction with Intel microcode updates.

When deploying a workload, these steps can help further protect workloads:

  1. Core scheduling: Isolating workloads on different physical cores reduces the attack surface for side channels. 
  2. Turn on hardware mitigations: Intel CPUs provide configurations that control speculative execution, such as Speculative Store Bypass Disable (SSBD).

Hardware-Software Collaboration for Microarchitectural Security

Intel believes that meeting the requirements for performance, user experience, and microarchitectural security requires hardware and software to work together to jointly implement practical security mechanisms. Software is in a unique position to provide hints to hardware to help enable secure and efficient program execution. 

On the hardware side, new generations of Intel products provide efficient in-silicon mitigations against a range of microarchitectural issues, such as Rogue Data Cache Load (RDCL), L1 Terminal Fault (L1TF) and Microarchitectural Data Sampling (MDS). Intel hardware addresses other microarchitectural issues that are impractical to directly mitigate in silicon, such as Speculative Store Bypass, by providing a variety of configurations and commands to allow software to choose   microarchitectural security controls according to the system’s threat model. For instance, the control register IA32_SPEC_CTRL_MSR provides a control to disable memory disambiguation speculation through Speculative Store Bypass Disable (SSBD), while the IA32_PRED_CMD_MSR provides a command called Indirect Branch Prediction Barrier (IBPB) that prevents code executed before the IBPB command from controlling the predicted targets of indirect branches after the IBPB command.  

Intel products also provide a Data Operand Independent Timing Instruction Set Architecture (DOIT ISA) for use in select security-sensitive workloads, such as implementations of cryptographic code.  (Intel Corp., Data Operand Independent Timing Instruction Set Architecture (ISA) Guidance, 2022). The DOIT ISA allows software to enter a data operand independent timing mode (DOITM) of execution wherein the execution time for all instructions on a specified list of data operand independent timing instructions is independent of the data values those instructions process. Using this mode helps software developers to write code which has execution time independent of the data values being operated on, which increases the resilience of code against timing side channels.

New ISA controls such as DOIT ISA and IA32_SPEC_CTRL_MSR empower software developers and system operators to tune microarchitectural security according to their threat model and performance requirements.  Different controls can have different performance impacts, and software may decide on a per-case basis which controls to enable. For instance, cryptographic implementations often demand for high security assurances and may thus use, among others, the DOIT ISA to optimize for those security assurances at the cost of some potential performance impact. On the other hand, a smaller subset of controls, such as IBRS, may be enabled for large parts of other software when speculative execution vulnerabilities are a concern. 

Intel believes that the most practical mechanisms to realize microarchitectural security are achieved by hardware and software working together. Besides tuning security via the aforementioned ISA controls, hardware can leverage more fine-grained software hints to more efficiently achieve microarchitectural security. Software can provide additional knowledge of the application domain and the code layout to help hardware appropriately control microarchitectural features. For instance, in Fine Indirect Branch Tracking (Fine IBT) (Moreira, 2021), with the help of the compiler, software provides information about legitimate indirect control-flow transfers to Intel Control Flow Enforcement Technology (Intel® CET) capable hardware in order to restrict speculation on illegitimate indirect branches and efficiently help prevent Branch Target Injection (BTI). 

Conclusion

Incidental channels such as covert channels and side channels exist in various forms in today’s computing systems. These incidental channels are mostly due to natural software coding patterns as well as hardware and software features that improve user experience, performance, and resource utilization. Incidental channels in hardware originate from data dependent timing optimizations, contention in shared resources, and changes to microarchitectural states. Additionally, transient execution adds another dimension to the applicability of incidental channels. Incidental covert channels allow attackers to bypass access control. As incidental channels are inherent to modern computer architectures that share resources among users, removing them is not feasible. To help protect against incidental covert channels, Intel’s focus is enabling software hardening itself to avoid covert channel attackers from gaining access to secrets. For incidental side channels, hardware provides the foundation for side channel resistant programming. Software should follow secure coding guidelines as well to help ensure that secret data is protected from being revealed via side channels in hardware. Intel strives to document relevant hardware behavior and is committed to provide safe default configurations for performance features. Intel’s approach of hardware and software jointly providing security mechanisms allows software to effeciently tune microarchitectural security to an application’s threat model and secure workloads against incidental channels.

Footnotes

  1. An operation might have data operand dependent timing and/or data operand independent timing. For example, a MUL instruction which loads a value from memory could have its execution time depend on the data operand, the memory address, or both. If the execution time of the MUL operation solely depends on the memory address, the MUL instruction is considered to have data operand independent timing. Irrespective of the MUL instruction‘s execution time being dependent on the memory address, if the MUL instruction’s execution time depends on its data operand, it is considered to have data operand dependent timing. 
  2. Note that speculation may cause data to be speculatively used as an address that software did not intend to access (for example, due to memory disambiguation). See Intel's documentation on transient execution attack mitigations for more details.

Bibliography

 

Software Security Guidance Home | Advisory Guidance | Technical Documentation | Best Practices | Resources

Product and Performance Information

1

Performance varies by use, configuration and other factors. Learn more at www.Intel.com/PerformanceIndex.