Visible to Intel only — GUID: GUID-4F1FD8E7-1879-4B89-8FEF-672848EBBD8E
Execution Model Overview
Thread Mapping and GPU Occupancy
Kernels
Using Libraries for GPU Offload
Host/Device Memory, Buffer and USM
Host/Device Coordination
Using Multiple Heterogeneous Devices
Compilation
OpenMP Offloading Tuning Guide
Multi-GPU, Multi-Stack and Multi-C-Slice Architecture and Programming
Level Zero
Performance Profiling and Analysis
Configuring GPU Device
Introduction
Performance Impact of Pinning GPU Frequency
Switching Intel® Xe-link On/Off
Sub-Groups and SIMD Vectorization
Removing Conditional Checks
Registerization and Avoiding Register Spills
Small Register Mode vs. Large Register Mode
Shared Local Memory
Pointer Aliasing and the Restrict Directive
Synchronization among Threads in a Kernel
Considerations for Selecting Work-Group Size
Prefetch
Reduction
Kernel Launch
Executing Multiple Kernels on the Device at the Same Time
Submitting Kernels to Multiple Queues
Avoiding Redundant Queue Constructions
Programming Intel® XMX Using SYCL Joint Matrix Extension
Doing I/O in the Kernel
Visible to Intel only — GUID: GUID-4F1FD8E7-1879-4B89-8FEF-672848EBBD8E
Configuring GPU Device
Introduction
For extracting best end-to-end performance from applications, users may need to sometimes modify device settings at a system level. These knobs may need to be applied by a system administrator, depending on whether sudo access is required.
This chapter provides guidelines and instructions for changing certain device settings for performance on Linux*.
Performance Impact of Pinning GPU Frequency
When applications use GPU for large portions of their computations, it’s recommended to pin the GPU at an optimal frequency. Some examples of these applications are High Performance Computing workloads which have computationally intensive portions in their algorithms offloaded to the device.
For applications that are memory bound, with kernels running for a very short time, but spend more time in data exchanges, the effects of pinning GPU to a higher frequency might be less pronounced or even detrimental.
The maximum fused GPU frequency, which is the theoretical HW maximum frequency, can be obtained using sysfs handles as noted below:
for (( i=1; i<$num_cards; i++ ))
do
for (( j=0; j<$num_tiles; j++ ))
do
cat /sys/class/drm/card$i/gt/gt$j/rps_RP0_freq_mhz;
done
done
Users can also obtain the current maximum software frequency, that can be dynamically modified by user, via rps_max_freq_mhz:
for (( i=1; i<$num_cards; i++ ))
do
for (( j=0; j<$num_tiles; j++ ))
do
cat /sys/class/drm/card$i/gt/gt$j/rps_max_freq_mhz;
done
done
Default policy in Linux kernel mode driver (i915) for server platforms is to set the frequency request range such that: rps_min_freq_mhz = rps_max_freq_mhz = rps_boost_freq_mhz = rps_RP0_freq_mhz
So for many workloads, this policy might provide best performance out of the box.
Users can set the frequency using below sysfs handles provided by the DRM Linux kernel driver.
for (( i=1; i<$num_cards; i++ ))
do
for (( j=0; j<$num_tiles; j++ ))
do
echo $desired_freq > /sys/class/drm/card$i/gt/gt$j/rps_min_freq_mhz;
echo $desired_freq > /sys/class/drm/card$i/gt/gt$j/rps_max_freq_mhz;
echo $desired_freq > /sys/class/drm/card$i/gt/gt$j/rps_boost_freq_mhz;
done
done
Notes regarding frequency pinning:
Firmware is final arbiter on granted frequency.
Some throttling may occur for thermal/power budget reasons.
Once frequency is pinned to a fixed value, there will be no dynamic scaling. * For server platforms current i915 policy pins frequency to rps_max_freq_mhz at boot time.
Switching Intel® Xe-link On/Off
Intel® Xe-link is a high-speed connectivity fabric hardware that provides accelerated data transfer capabilities for scale-up and scale-out operations. These are typically used for inter-GPU and inter-node data transfer operations for HPC applications deployed at cluster scale. However, for applications that don’t use these capabilities, it could be beneficial to turn off the power to this resource selectively to allow for lower frequency throttling on the GPU compute engines.
Following command lines describe how to turn off power to Intel® Xe-link:
modprobe -r iaf;
for i in {0..$num_cards}; do
cat /sys/class/drm/card$i/iaf_power_enable;
done;
for i in {0..$num_cards}; do
echo 0 > /sys/class/drm/card$i/iaf_power_enable;
done;
for i in {0..$numcards}; do
cat /sys/class/drm/card$i/iaf_power_enable;
done
$num_cards can be obtained by listing /sys/class/drm/ directory and noting how many card* sub-directories exist.