Intel® VTune™ Profiler

Cookbook

Download PDF
ID 766316
Date 6/24/2024
Public
Document Table of Contents
Document Table of Contents x
Cookbook
Cookbook x
Methodologies Configuration Recipes Tuning Recipes Notices and Disclaimers
Methodologies x
Analyzing Uncore Perfmon Events for Use of Intel® Data Direct I/O Technology in Intel® Xeon® Processors (NEW) Top-down Microarchitecture Analysis Method OpenMP* Code Analysis Method Custom Data Collection for Performance Analysis (NEW) Software Optimization for Intel® GPUs (NEW) Core Utilization in DPDK Apps PCIe Traffic in DPDK Apps DPDK Event Device Profiling Effective Utilization of Intel® Data Direct I/O Technology Compile a Portable Optimized Binary with the Latest Instruction Set
Configuration Recipes x
Profiling OpenVINO™ Applications (NEW) Ingredients Set up OpenVINO™ Build the OpenVINO™ source Configure OpenVINO™ for Performance Analysis Profile CPU Hot Spots Profile GPU Hot Spots Profile NPUs See Also Profiling High Bandwidth Memory Performance on Intel® Xeon® CPU Max Series (NEW) Profiling Windows* Applications for Hybrid CPU Platforms (NEW) Viewing Analysis Results on a Web Browser (NEW) Profiling Machine Learning Applications (NEW) Profiling Single-Node Kubernetes* Applications (NEW) Analyzing Hot Code Paths Using Flame Graphs (NEW) Improving Hotspot Observability in a C++ Application Using Flame Graphs Measuring Performance Impact of NUMA in Multi-Processor Systems Profiling Games built with Unity* (NEW) Profiling Games built with Unreal Engine* (NEW) Profiling Java Applications as a Remote User (NEW) Profiling JavaScript* Code in Node.js* Analyzing CPU and FPGA (Intel® Arria® 10 GX) Interaction Profiling a .NET* Core Application Profiling Applications in Amazon Web Services* (AWS) EC2 Instances Enabling Performance Profiling in GitLab* CI Configuring a Hyper-V* Virtual Machine for Hardware-Based Hotspots Analysis Profiling an Application for Performance Anomalies (NEW) Profiling an OpenMP* Offload Application running on a GPU Profiling a SYCL* Application running on a GPU Profiling an FPGA-driven SYCL* Application Profiling Hardware Without Intel Sampling Drivers Profiling MPI Applications Profiling Docker* Containers Profiling a Remote Target Through a Proxy Server (NEW) Profiling in an Apptainer* Container Profiling Linux*, Android*, and QNX* System Boot Time Using Intel® VTune™ Profiler Server with Visual Studio Code and Intel® DevCloud for oneAPI (NEW) Using Intel® VTune™ Profiler Server in HPC Clusters Using the Command-Line Interface to Analyze the Performance of a SYCL* Application running on a GPU (NEW)
Tuning Recipes x
Cache-Related Latency Issues in Segmented Cache Environment False Sharing Frequent DRAM Accesses Poor Port Utilization Page Faults Instruction Cache Misses OS Thread Migration OpenMP* Imbalance and Scheduling Overhead Processor Cores Underutilization: OpenMP* Serial Time Scheduling Overhead in an Intel® oneAPI Threading Building Blocks Application PMDK Application Overhead
Cookbook

Profiling OpenVINO™ Applications (NEW)

Learn how to use Intel® VTune™ Profiler to profile AI applications. This recipe uses a benchmark application in the OpenVINO™ toolkit.

Content Expert: Kali Uday Balleda, Kumar Sanatan

  • INGREDIENTS

  • DIRECTIONS:

    1. Set up OpenVINO™.

    2. Build the OpenVINO™ source.

    3. Configure OpenVINO™ for Performance Analysis.

    4. Profile CPU Hot Spots.

    5. Profile GPU Hot Spots.

    6. Profile NPUs.

Ingredients

Here are the hardware and software tools you need for this recipe.

Set up OpenVINO™

To set up OpenVINO™ from the package manager, install these applications in sequence:

  1. If you use oneDNN or oneDAL libraries, install the Intel® oneAPI Base Toolkit. Follow these instructions.
  2. Next, install Intel® Distribution for Python* (Intel Python3).
  3. Set up CMake.
  4. Next, set up Microsoft Visual Studio 2022.
  5. Install Git for Windows.
  6. At the command prompt, set the path to Intel Python3. In the folder where you downloaded Intel Distribution for Python, run vars.bat from the env subdirectory.
  7. Install the latest version of the OpenVINO™ toolkit: 
    pip install OpenVINO™

Build the OpenVINO™ source

  1. In Github, clone the OpenVINO™ repository: 
    git clone https://github.com/OpenVINO™toolkit/OpenVINO™.git
  2. Open the OpenVINO™ folder: 
    cd OpenVINO™
  3. Update the submodule: 
    git submodule update –init
  4. Create a build directory: 
    mkdir build
  5. Open the build directory: 
    cd build
  6. Generate a Microsoft Visual Studio project with the OpenVINO source code. Make sure to enable the OpenVINO runtime with Python API. Also enable the ability to profile with the Instrumentation and Tracing Technology (ITT) API. 
    cmake -G “Visual Studio 17 2022” <path\to\OpenVINO™ source code> -DCMAKE_BUILD_TYPE=Release 
-DENABLE_PYTHON=ON  -DPython3_EXECUTABLE= "C:\Users\sdp\Downloads\intelpython3\python.exe" 
-DENABLE_PROFILING_ITT=ON
  7. Open the OpenVINO™.sln solution in Microsoft Visual Studio 2022.
  8. Build the solution file in release mode.

When the build is successful, you find a new bin directory that contains:

  • Python libraries in bin\intel64\Release\python
  • OpenVINO™ Plugins and DLLs in bin\int64\Release\

Configure OpenVINO™ for Performance Analysis

Before you run performance analyses on AI applications, make sure to configure OpenVINO™ first:

  1. Configure the OpenVINO™ build with the Instrumentation and Tracing Technology API (ITT API). Set these environment variables: 
    set PYTHONPATH=<path to OpenVINO™ repository>\bin\intel64\Release\python;%PYTHONPATH%
set OpenVINO™_LIB_PATHS=<path to OpenVINO™ repository>\bin\intel64\Release;%OpenVINO™_LIB_PATH%
  2. Run the setup script for Python: 
    C:\Users\sdp\intelpython3\env\vars.bat
  3. Run the setup script for oneAPI tools: 
    C:\Program Files (x86)\Intel\<toolkit_version>\oneAPI-vars.bat
Understand the Sample Application:

Before you run an AI application on an Intel processor, you must convert the AI model into the OpenVINO™ IR format. To learn more about this conversion, use the omz_converter tool.

You can run the profiling analyses described here on any AI application of your choice. This recipe uses the Benchmark tool (available in the OpenVINO™ toolkit) to demonstrate how you run analyses with VTune Profiler. Use benchmark_app to calculate the throughput and run a latency analysis for your AI application.

To run benchmark_app, at the command prompt, type this command: 

benchmark_app -m model.xml -d NPU -niter 1000

where:

  • model.xml is your AI application

  • d is the target device (CPU/GPU/NPU)

  • niter is the number of iterations

Profile CPU Hot Spots

To identify CPU bottlenecks, run the Hotspots analysis . In the command window, type: 

vtune -c hotspots -r <path to VTune results directory> -- benchmark_app -m model.xml -d CPU -niter 1000

Open the result of the analysis and identify CPU hot spots in the Bottom-up and Platform windows.

Profile GPU Hot Spots

To identify GPU bottlenecks, run the GPU Compute/Media Hotspots analysis . In the command window, type: 

vtune -c gpu-hotspots -r <path to VTune results directory> -- benchmark_app -m model.xml -d GPU -niter 1000

Open the result of the analysis and identify GPU hot spots in the Graphics window.

Profile NPUs

To profile NPUs, run the NPU Exploration analysis . In the command window, type: 

vtune -c npu -r <path to VTune results directory> -- benchmark_app -m model.xml -d npu -niter 1000

Open the result of the analysis and understand NPU usage in the Platform window.

Parent topic: Configuration Recipes
Level Two Title