Intel® Xe Super Sampling 2 Whitepaper

This whitepaper describes X^eSS 2, Intel’s latest set of rendering technologies for efficient built-in GPUs and powerful discrete GPUs. 

Intel® X^e Super Sampling 2  

X^eSS 2 introduces two groundbreaking features: X^eSS Frame Generation (X^eSS-FG) and X^e Low Latency (X^eLL). Combined with X^eSS Super Resolution (X^eSS-SR), developers now have more tools to improve visual quality, performance and responsiveness of modern games. All X^eSS 2 games will support these three technologies.

X^eSS Frame Generation dramatically improves gaming smoothness by achieving higher frame rates through AI-based frame interpolation. Meanwhile, X^e Low Latency (X^eLL) ensures a more responsive and immersive experience by reducing the delay between your actions and the on-screen response. Together, these advancements mark a significant leap forward, delivering unparalleled smoothness and responsiveness for gamers everywhere.

Why do we need frame interpolation?

 In an ideal world, each frame a game renders would be produced at the exact same delta time (dt) interval and at fast framerates:

This consistency and speed are ideal for game physics simulation, animation, and presentation, providing the smoothest experience for users. However, in practice, games are dynamic system that simultaneously manages many different subsystems, requiring a variable amount of time to produce each frame. Additionally, long frame times will create scenarios of low animation smoothness as the difference between frame animations become more pronounced.

This all leads to an undesirable perceived stutter and sluggish gameplay.

One straightforward solution is to maximize the frame rate by reducing game setting, which minimizes any delta time. As the time to render each frame decreases, variations become less noticeable. In practice, a game will eventually reach its maximum performance on a given platform. Additionally, as the settings are lowered, the visual quality of the game becomes less immersive.

Introducing X^eSS Frame Generation

With the introduction of X^eSS Frame Generation (X^eSS-FG), Intel offers developers another way to improve frame rendering smoothness and frame rate: frame interpolation. This technique helps achieve a more consistent and fluid gaming experience.

The diagram above illustrates how X^eSS FG operates, featuring two main components:

1. Frame Interpolation: X^eSS-FG creates an interpolated frame between two consecutive original frames. This interpolation takes a fixed amount of time, depending on the rendering resolution, and is optimized to run much faster than original frame renders.
2. Order of Presentation: Once an interpolated frame is ready, X^eSS-FG changes the order of presentation by showing the interpolated frame first and delaying the display of the game’s original frame. Intel’s display driver continuously measures how long each frame takes to render and schedules frames to be presented at approximately half that delta time.

As a result of these two steps user can expect the following:

1. Increased Frame Rate: The exact scaling formula considers the average frame rate of a given game and the time it takes to perform frame interpolation.
2. Smoother Frame Intervals: Frames are presented at more frequently, reducing any frame rendering spikes. For example, if the original game had the largest delta time between presented frames as dt3, with frame generation it becomes (dt3’-dt2’/2).

How does X^eSS Frame Generation work?

Using state-of-the-art AI models, X^eSS-FG creates a new high quality frame between two consecutive original frames:

X^eSS-FG works with the game engine to access rich input data with motion vectors and depth buffer feeding in the technology on top of the frame renders. Through a process called motion vector based reprojection, using game engine movement data for visible objects, we compute a first set of intermediate candidates for the interpolated frame.

We generate a second set of intermediate candidates with an AI model trained for optical flow reprojection. Optical flow evaluates movement based on the pixel difference between 2 frames and gives key information on movement for visual elements, such as shadows, reflections or particles that are not treated as objects from the game engine perspective.

A second AI model with a blend function evaluate both sets of candidate frames and pixels to reconstruct a high-quality interpolated frame.

Finally, the UI is composited on top of the frame. X^eSS-FG offers developers a few options for UI composition, including repeating UI or rendering a new UI for the interpolated frame.

What are the benefits of X^eSS-FG?

X^eSS 2 delivers a large performance improvement over the first generation. The integration of Frame Generation (FG) with X^eSS-SR significantly enhances performance in F1'24 at 1440p Ultra High settings with Ray Tracing (RT) enabled. Across various Super Resolution (SR) modes, X^eSS 2 delivers up to 3.9x scaling compared to native and up to 1.7x scaling compared to X^eSS-SR alone:

What causes latency in games? 

1. An input device is the first step in displaying an updated image. The device we use has a physical restriction on how frequently it can receive user inputs, known as the polling rate.
   • A typical wired mouse and keyboard can have a polling rate between 1ms to 20ms. Wireless input devices usually have higher latency compared to their wired counterparts.
2. The game's CPU threads poll for user inputs. Games vary in their latency management systems but typically, many poll for an input once at the beginning of frame rendering. User input influences the game's virtual camera position, which then feeds into the game's logic to prepare visible objects for rendering, logic, and physics.
3. Once the CPU has completed its logic, the game's render thread processes all the data needed for rendering and issues API commands using a 3D API, such as DirectX 12, Vulkan, or others.
4. These API commands compose a render queue, which may need to wait before it can get onto the GPU, typically if GPU is rendering the previous frame.
5. The GPU renders the frame and that also takes time. A frame is considered complete when it is "presented," meaning a command is issued to display it on the physical screen.
6. Presentation can be done in a few different ways, but for the purpose of this guide we’ll simplify it to a few things:
   • VSync ON: a frame will wait for the display to perform vertical synchronization. Fixed Refresh Rate (FRR) monitors can only synchronize at fixed intervals, such as every 16.6 ms for a 60 Hz monitor. Variable Refresh Rate (VRR) monitors can synchronize within a range of values. For example, a 240 Hz VRR monitor with a minimum refresh rate of 30 Hz can synchronize no faster than 4 ms after the previous frame but can also synchronize within a range of 4-33 ms.
   • VSync OFF: a frame can be pushed to display immediately, but user may see tearing on the screen.
7.  A display has its own physical properties that determine how it lights up its light-emitting elements.
8. Measuring system latency would require additional tooling that could observe change in the pixels based on input. Latency Measuring Tool (LMT) is a device like that, that combines a photo sensor and additional controls to capture how quickly pixels change based on luminance.   

The application loop described above has been refined over time to prioritize maximum FPS. The accumulated latency between stages 2-6 allows different subsystems (game, OS, drivers, GPU) to absorb any unexpected spikes in processing time that might occur in any one area. While this highly pipelined system generally performs well, there are instances where users may prefer to prioritize responsiveness over maximum frame rate.

Introducing X^e Low Latency and Driver Low Latency

Intel’s X^e Low Latency (X^eLL) and Driver Low Latency are technologies built to reduce latency in games. They are designed to work in various modes described below:

1. Support for various frame presentation modes, including VSync On, VSync Off, and VRR modes. Note that enabling VSync also implicitly sets an FPS cap.
2. Support with frame rate limiters.

Intel’s X^e Low Latency is designed to reduce latency while maintaining current FPS as much as possible. The most significant latency savings are achieved in GPU-bound scenarios or when paired with VSync ON or frame rate limiters.

How does X^eLow Latency work?

Intel’s X^e Low Latency (X^eLL) and Driver Low Latency Mode can optimize various steps between 2-6:

Step 2: X^eLL is integrated directly into applications, instructing games to sleep before collecting user inputs. Driver Low Latency Mode targets older games, which typically collect new user inputs after the previous frame has been rendered. By delaying the input collection, both technologies bring the input collection and final frame presentation closer together, enhancing responsiveness.

Step 3 and 4: X^eLL calculates the time it takes to render a frame on the CPU and GPU and unblocks the CPU work only when it can let GPU and CPU work on a frame in parallel. The time a frame waits in a render queue is removed completely and CPU time is largely hidden behind the GPU rendering time.

Step 5: When a user enables a frame rate limit or VSync ON, the GPU may take longer to render the frame. This is a desired behavior because the Dynamic Power Manager (DPM) ensures that the system balances performance, temperature, and power consumption. With Latency Mode ON, the Intel graphics driver prompts the DPM to respond for a specific application. With Latency Mode ON+Boost mode, the driver sends additional hints to the power manager to prioritize high performance and fast power response for a given process. This can be especially useful on battery-powered devices, where the DPM typically takes a more proactive role in extending battery life.

Step 6: For Displays with Fixed Refresh rates, and games running with VSync ON, X^eLL and Driver Low Latency Mode enhance the experience by ensuring that frames are rendered as close to the vertical synchronization as possible. The Intel driver continuously tracks when the frame is ready to be presented and how long it waits to be displayed. It then influences the application to start at the optimal moment to minimize this wait time.

What are the benefits of X^eLL?

Intel’s X^e Low Latency (X^eLL) is designed to reduce the time between user input and the rendered result appearing on the screen. Here's how it compares:

• Baseline Improvement (grey bar vs. dark purple bar): X^eLL reduces latency up to 45% compared to the standard game rendering process.
• FPS Improvement (grey bar vs. dark blue bars): X^eSS Super Resolution (SR) improves latency by increasing frame rate.
• X^eLL Paired with X^eSS-SR (dark blue bars vs. light blue bars): when X^eLL is paired with SR, it further reduces latency in games.
• X^eSS 2 Combined Improvements (dark purple bar vs. light purple bars): when all technologies (SR, LL, FG) are used together , the solution we call X^eSS 2, gamers experience a significant boost in frame rate (up to 3.9x) and with better latency than the game running without any X^eSS technology.

Enabling XeLL or Driver Low Latency Mode

X^eLL is enabled in games supporting X^eSS 2. Developers are encouraged to expose X^eLL as an individual toggle for use with X^eSS-FG and as a standalone feature. Below is a UI mockup:

Driver Low Latency Mode is enabled in Intel Graphics Software for DirectX 11 and DirectX 9 games. To enable, please go to Graphics -> Frame Delivery -> Low Latency Mode. Below is a screenshot of Low Latency Mode enabled.

X^eSS 2 Support Matrix

X^eSS 2 is designed to be compatible with a wide range of hardware. Below are the requirements for each sub-feature:

• X^eSS Super Resolution (SR): Supported on all X^e+ graphics platforms. X^eSS-SR also supports cross-vendor platforms with support for SM 6.4 capability or newer.
• X^e Low Latency (LL): Available on all X^e-xPG graphics platforms and newer, leveraging app and driver latency improvements for enhanced responsiveness.
• X^eSS Frame Generation (FG): Utilizes complex AI models, requiring XMX hardware acceleration. Frame generation is supported on all Intel Arc GPUs with XMX hardware acceleration.

X^eSS 2 is compatible   with the latest game technologies. More specifically, the X^eSS-FG and X^eLL APIs are designed for DirectX 12. Additionally, we offer a driver mode Low Latency feature in Intel Graphics Software that supports DirectX 11 and DirectX 9. Recently, X^eSS-SR has expanded its support to include DirectX 11, DirectX 12, and Vulkan, providing even greater flexibility for developers.

Workloads and Configurations

Data gathered for this guide was collected from Nov 13-25:

• Motherboard: ASUS ROG MAXIMUS Z790 HERO, BIOS Version: 2703
• CPU: Intel® Core™ i9-14900K
• Memory: Corsair DOMINATOR PLATINUM RGB DDR5 32GB (2x16GB) 5600MHz C36 (CMT32GX5M2B5600C36)
• Storage: Corsair MP600 PRO XT (NVMe)
• Power Supply: Corsair RMX Series (2021), RM1000x 1000W

• OS Version: Windows 11 Pro (26100.2033)
• Power Plan: Best Performance

• Windows Defender: Enabled
• VBS: Enabled
• Intel VT-d: Enabled
• Resizable BAR: Enabled

GPU configurations listed below:

Important Notices and Legal Disclaimers

Performance varies by use, configuration, and other factors. Learn more at www.Intel.com/PerformanceIndex. 

Intel technologies may require enabled hardware, software, or service activation.

No product or component can be absolutely secure.

Your costs and results may vary.

Intel technologies may require enabled hardware, software, or service activation.

No license (express or implied, by estoppel or otherwise) to any intellectual property rights is granted by this document.

The products described may contain design defects or errors known as errata which may cause the product to deviate from published specifications.  Current characterized errata are available on request.

Altering clock frequency or voltage may void any product warranties and reduce stability, security, performance, and life of the processor and other components.  Check with system and component manufacturers for details.

AI features may require software purchase, subscription or enablement by a software or platform provider, or may have specific configuration or compatibility requirements. Learn more at intel.com/aipc. Performance varies by use, configuration and other details.

Results that are based on pre-production systems and components as well as results that have been estimated or simulated using an Intel Reference Platform (an internal example new system), internal Intel analysis or architecture simulation or modeling are provided to you for informational purposes only. Results may vary based on future changes to any systems, components, specifications or configurations.

© Intel Corporation.  Intel, the Intel logo, and other Intel marks are trademarks of Intel Corporation or its subsidiaries.  Other names and brands may be claimed as the property of others.