Prerequisites

If you intend to debug on a GPU, first check whether your device is supported for debugging kernels offloaded to GPU by checking the list of supported accelerators. If your GPU device is not listed above, then a breakpoint inside the kernel won’t be hit. In that case, you can still debug the offload to CPU.

Install the latest GPU drivers and configure your system to use them. Refer to the Intel® oneAPI Toolkits Installation Guide for Linux* OS. Follow the instructions Install Intel GPU Drivers to install GPU drivers matching your system.

You can also install an extension for Visual Studio Code* for debugging GPU with Intel Distribution for GDB. Refer to the Using Visual Studio Code with Intel® oneAPI Toolkits Guide.

Set Up the GPU debugger

To set up the GPU debugger, you must have root access.

1. If you intend to debug on a GPU, a Linux Kernel that supports GPU debugging is needed.

   1. Follow the instructions at Intel® software for general purpose GPU capabilities to download and install the necessary drivers.

   2. Enable i915 debug support in Kernel:

      1. Open a terminal.

      2. For each GPU, set the value of the prelim_enable_eu_debug flag to 1. You can use the following shell command for this:

         for f in /sys/class/drm/card*/prelim_enable_eu_debug; do echo 1 > "$f"; done

         NOTE:

         System admins may have to give write permissions to the prelim_enable_eu_debug entry in the sysfs for regular users to be able to enable/disable debug.

   3. If you want to enable i915 debug support in kernel persistently, follow the instructions below (requires admin permissions):

      1. Open a terminal.

      2. Open the grub file in /etc/default.

      3. In the grub file, find the line GRUB_CMDLINE_LINUX_DEFAULT="".

      4. Enter the following text between the quotes (""):

         i915.debug_eu=1

         NOTE:

         By default, the GPU driver does not allow workloads to run on a GPU longer than a certain amount of time. The driver kills such long-running workloads by resetting the GPU to prevent hangs. The hangcheck mechanism of the driver is disabled if the application is running under the debugger. If you plan to run long compute workloads also without a debugger being attached, consider applying GPU: Disable Hangcheck by adding

         i915.enable_hangcheck=0

         to the same GRUB_CMDLINE_LINUX_DEFAULT line.

      5. Update GRUB for these changes to take effect:

         sudo update-grub

      6. Reboot.

2. Setup your oneAPI CLI environment. The environment setup examples shown below assume you are using the “component” directory layout. If you wish to use the “unified” directory layout, see the oneAPI Development Environmental Setup for details.

   Linux (sudo):

   source /opt/intel/oneapi/setvars.sh

   Linux (user):

   source ~/intel/oneapi/setvars.sh

3. Setup environment

   Use the following environment variable to enable debugger support for Intel® oneAPI Level Zero:

   export ZET_ENABLE_PROGRAM_DEBUGGING=1

4. System check

   When everything is ready, please run the following command to confirm that the system configuration is reliable:

   python3 /path/to/intel/oneapi/diagnostics/latest/diagnostics.py --filter debugger_sys_check --force

   A possible output of a well configured system is as follows:

   ...
   Checks results:
   
   ===========================================================================================================
   Check name: debugger_sys_check
   Description: This check verifies if the environment is ready to use gdb (Intel(R) Distribution for GDB*).
   Result status: PASS
   Debugger found.
   libipt found.
   libiga found.
   i915 debug is enabled.
   Environmental variables correct.
   ===========================================================================================================
   
   1 CHECK: 1 PASS, 0 FAIL, 0 WARNINGS, 0 ERRORS
   
   Console output file: /path/to/logs/diagnostics_filter_debugger_sys_check_force.txt
   JSON output file: /path/to/diagnostics/logs/diagnostics_filter_debugger_sys_check_force.json
   ...

Compile the program with debug information

You can use the sample project, Array Transform, to quickly get started with the application debugger.

1. To get the sample, choose any of the following ways:

   • Use the oneAPI CLI Samples Browser to select Array Transform from the Getting Started category.

   • Download from GitHub*.

2. Navigate to the src of the sample project:

   cd array-transform/src

3. Compile the application by enabling the debug info (-g flag) and turning off optimizations (-O0 flag). Disabling optimization is recommended for a stable and accurate debug environment. This helps to avoid confusion caused by changes to the code after compiler optimizations.

   NOTE:

   You can still compile the program with optimization enabled (-O2 flag), which can be helpful if you aim at GPU assembly debugging.

   You can compile the program in several ways. Options 1 and 2 use just-in-time (JIT) compilation, which is recommended to debug the sample. Option 3 uses ahead-of-time (AOT) compilation.

   • Option 1. You can use the CMake file to configure and build the application. Refer to the README of the sample for the instructions.

     NOTE:

     The CMake file provided with the sample already passes the -g -O0 flags.

   • Option 2. To compile the array-transform.cpp sample application without the CMake file, issue the following commands:

     icpx -fsycl -g -O0 array-transform.cpp -o array-transform

     If compilation and linking is done separately, retain the -g -O0 flags at the link step. The link step is when icpx translates these flags to be passed to the device compiler at runtime. Example:

     icpx -fsycl -g -O0 -c array-transform.cpp
     icpx -fsycl -g -O0 array-transform.o -o array-transform

   • Option 3. You can use AOT compilation to avoid longer JIT compilation times at runtime. JIT compilation can take significantly longer for large kernels under the debugger. To use Ahead-of-Time compilation mode:

     • For debugging on a GPU:

       Specify the device that you will use for program execution. For example, -device dg2-g10 for Intel® Data Center GPU Flex 140 Graphics. For the list of supported options and more information on AOT compilation, refer to the Intel® oneAPI DPC++ Compiler Developer Guide and Reference.

       For example:

       icpx -fsycl -g -O0 -fsycl-targets=spir64_gen -Xs "-device dg2-g10" array-transform.cpp -o array-transform

       Ahead-of-Time compilation requires the OpenCL™ Offline Compiler (OCLOC). For more information refer to the section titled “Install OpenCL™ Offline Compiler (OCLOC)” in the Installation Guide.

     • For debugging on a CPU:

       icpx -fsycl -g -O0 -fsycl-targets=spir64_x86_64 array-transform.cpp -o array-transform

Start a debug session

Start the debug session:

1. Start Intel Distribution for GDB as follows:

   gdb-oneapi array-transform

   You should see the (gdb) prompt.

2. You can control the device that is used to offload the kernel by defining the environment variable ONEAPI_DEVICE_SELECTOR. From the debugging session you can modify it with set env command.

   • For debugging on a CPU device:

     set env ONEAPI_DEVICE_SELECTOR=*:cpu
     run

     Example output:

     [SYCL] Using device: [Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz] from [Intel(R) OpenCL]

   • For debugging on the Level Zero GPU device:

     set env ONEAPI_DEVICE_SELECTOR=level_zero:gpu
     run

     NOTE:

     The Level Zero GPU device is the default choice when it is available and the variable is not set.

     Example output:

     [SYCL] Using device: [Intel(R) Data Center GPU Flex Series 140 [0x56c1]] from [Intel(R) Level-Zero]

   • For debugging on the FPGA-emulator:

     set env ONEAPI_DEVICE_SELECTOR=*:fpga
     run

     Example output:

     [SYCL] Using device: [Intel(R) FPGA Emulation Device] from [Intel(R) FPGA Emulation Platform for OpenCL(TM) software]

3. To quit the Intel Distribution for GDB:

   quit

For your convenience, common Intel Distribution for GDB commands are provided in the Reference Sheet.

To debug the Array Transform sample and learn more about Intel Distribution for GDB, walk through basic debugging scenarios using the Tutorial.

Learn More

Notices and Disclaimers

Intel technologies may require enabled hardware, software or service activation.

No product or component can be absolutely secure.

Your costs and results may vary.

© Intel Corporation. Intel, the Intel logo, and other Intel marks are trademarks of Intel Corporation or its subsidiaries. Other names and brands may be claimed as the property of others.

No license (express or implied, by estoppel or otherwise) to any intellectual property rights is granted by this document.

The products described may contain design defects or errors known as errata which may cause the product to deviate from published specifications. Current characterized errata are available on request.

Intel disclaims all express and implied warranties, including without limitation, the implied warranties of merchantability, fitness for a particular purpose, and non-infringement, as well as any warranty arising from course of performance, course of dealing, or usage in trade.

OpenCL and the OpenCL logo are trademarks of Apple Inc. used by permission by Khronos.