Using Intel® Distribution for GDB with Docker*
This example describes how to debug Docker* applications running on Intel GPU devices. Instructions how to debug natively can be found here. It describes how to create and use Docker images for the following debugging scenarios, and how to use the run image command to debug.
This example explains how to use Using Intel® Distribution for GDB (gdb-oneapi) with Docker containers based on the ubuntu:latest image. The same approach can be used to debug in images based on other supported GNU/Linux distributions.
NOTE:
This example describes some configuration options that reduce the security of your containers. These should NOT be used in production environments and are intended for debugging purposes only.
There are several ways to configure debugging inside containers. For now, use the following scenarios:
gdb-oneapi and an application to be debugged in a single container.
gdb-oneapi and a server in one container, with the application to be debugged in a second container.
There are different prerequisites for each scenario mentioned above.
Docker Installation
See Docker Engine Overview and Getting Started with Docker for more details.
A Docker container isolates a related collection of applications and their dependencies (libraries, environment variables, configuration files, scripts, etc.) from other containers and processes that are running on the same Linux* host system. However, all Docker container images running on a specific Linux host share the underlying Linux kernel and drivers. Thus, oneAPI drivers and kernel modules need to be installed and configured into the Linux system that is hosting the Docker containers for the container applications to be able to access those host machine resources. To allow debugging you must do the following:
Install packages with the custom kernel and kernel modules.
Set all required boot arguments to kernel. For more details see the Installation Guide
NOTE:
When pulling images from Docker Hub you might reach the pull request rate limit. You can create an account to avoid the problem. More information can be found here. After creating an account and verifying your e-mail, run docker-login on your host machine.
Example Application
Use the array-transformexample as the target application. Compile with optimization flags -O0 -g.
Modify the example application and add following line after main():
for (int i = 1; i < argc; ++i) if (strcmp(argv[i], "wait") == 0) getchar();This causes our application to wait for user input if the wait parameter was passed.
Docker Container
Official Docker images for Intel® oneAPI Base Toolkit (Base Kit) can be found here. Alternatively, the following Dockerfile can be used:
FROM ubuntu:latest
RUN DEBIAN_FRONTEND=noninteractive \
TZ=Europe/Berlin \
apt-get update && \
apt-get install -y \
tzdata wget gpg
RUN wget -O- https://apt.repos.intel.com/intel-gpg-keys/GPG-PUB-KEY-INTEL-SW-PRODUCTS.PUB \
| gpg --dearmor \
| tee /usr/share/keyrings/oneapi-archive-keyring.gpg \
> /dev/null
RUN echo "deb [signed-by=/usr/share/keyrings/oneapi-archive-keyring.gpg] \
https://apt.repos.intel.com/oneapi all main" | \
tee /etc/apt/sources.list.d/oneAPI.list
RUN apt-get update && \
apt-get install -y intel-basekit
RUN echo 'source /opt/intel/oneapi/setvars.sh --force' > $HOME/.bashrc
ENV ZET_ENABLE_PROGRAM_DEBUGGING=1
IMPORTANT:
The oneAPI evironment setup shown above assumes you are using the “component” directory layout. If you wish to use the “unified” directory layout, see the oneAPI Development Environmental Setup for details.
Debug in a Single Container
In this basic use case, all of your development and debugging is happening in a single container. Create a Dockerfile named Dockerfile.all-in-one with content from the previous section.
Now you can build the image in the same directory where your Dockerfile is. Run the following command:
$ docker build -t intel-one-api-all-in-one -f Dockerfile.all-in-one $(pwd)Building the image for the first time might take a longer time, but after this step you will have a Docker image called intel-one-api-all-in-one.
Then run the image and work in an isolated environment:
$ docker run \
-it \
--device /dev/dri \
-v <example-source-path-on-host>:/sources \
intel-one-api-all-in-oneThe array-transformexample should now be in the /sources directory in the container. You can build the example in the container by following the build instructions in the example README file.
Before starting a debug session, we suggest running sycl-ls to see available hardware. An example output is shown below:
> $ sycl-ls
> ...
> [ext_oneapi_level_zero:gpu:0] Intel(R) Level-Zero, Intel(R) Graphics [0x56c1] 1.3 [1.3.0]
> [ext_oneapi_level_zero:gpu:1] Intel(R) Level-Zero, Intel(R) Graphics [0x56c1] 1.3 [1.3.0]
> ...After building the application sources, you can start debugging. Run inside the container:
$ gdb-oneapi /sources/<path-to>/array-transform -ex "set args gpu" -ex "b GetDim"Specifying -ex "set args gpu" sets the application argument to “gpu” so the test application uses the “gpu” device, which you should have seen in the sycl-ls output
The option -ex "b GetDim" puts a breakpoint in function GetDim.
NOTE:
The gpu argument is specific to the array-transform application.
Inside the Intel Distribution for GDB, you can start the debug session with (gdb) run.
Debug in Two Containers
In this scenario, the target application executes in a separate deployment container. The goal is to debug it remotely from a development container with gdb-oneapi.
Both containers should be in the same namespace. Assume the deployment container has the name “isolated_app_to_debug_container” (Run docker container list to get the actual name of the container with the target application running in it).
NOTE:
This container must run with --device /dev/dri to offload code to a GPU.
Run modified array-transform as follows:
$ docker run -it --rm \
--name isolated_app_to_debug_container \
--device /dev/dri \
-v <host-app-exec-path>:<isolated-target-app-exec-path> \
intel-one-api-official \
<isolated-target-app-exec-path> gpu wait
NOTE:
-v <host-app-exec-path>:<isolated-target-app-exec-path> is not required. Target executable can be located in container. If it is isolated, then <isolated-target-app-exec-path> gpu wait should be replaced with a command to start target application.
This starts the target application, which waits for an input (if you made the modifications to the example app, as described in Example application).
To start the Intel Distribution for GDB session, run the following command:
$ docker run -it \
--cap-add SYS_ADMIN \
--cap-add SYS_PTRACE \
--device /dev/dri \
--security-opt seccomp=unconfined \
-v <sources-path-on-host>:<sources-path-in-container> \
--pid=container:isolated_app_to_debug_container \
--network=container:isolated_app_to_debug_container \
intel-one-api-official \
gdb-oneapi -p 1 -ex "dir <sources-path-in-container>" -ex "b GetDim"
NOTE:
To debug the target application when Intel Distribution for GDB is also containerized, the debugger needs to have access to the binary code and optional access to the directory with the target application’s source code. Also, additional privileges must be granted to the container with Intel Distribution for GDB (see --cap-add SYS_ADMIN --cap-add SYS_PTRACE parameters)
With the above command, you will see a warning from Intel Distribution for GDB: warning: Error disabling address space randomization: Operation not permitted. This happens because GDB by default tries to disable ASLR. To disable it, refer to this article .
After this command, Docker starts Intel Distribution for GDB, which will be attached to process 1. The previous command makes the target application reachable in the new newly created container by PID 1. Also, -ex "dir <sources-path-in-container>" specifies the directory with source files to fix file name mapping. The -ex "b GetDim" puts a breakpoint on function GetDim(). This can be skipped because breakpoints can be added later. Please note that the command starts the debugger, which attaches to and pauses the target application.
Remarks
Since PID’s of containerized applications are accessible from the host machine as an optional scenario, the target application can run in the container, and the debugger can run using this command:
$ gdb-oneapi -p $(pgrep <isolated-target-app-exec-path>)``.However, we do not recommend this approach because to get access to a target application process Linux KernelHardening should be disabled. The better solution would be to expose a port and use it in combination with gdbserver-ze.