Intel® Advisor User Guide

Download PDF
ID 766448
Date 7/13/2023
Public

A newer version of this document is available. Customers should click here to go to the newest version.

Document Table of Contents
Document Table of Contents x
Intel® Advisor User Guide
Intel® Advisor User Guide x
Introduction Install and Launch Intel® Advisor Set Up Project Analyze Vectorization Perspective Analyze CPU Roofline Model Threading Designs Model Offloading to a GPU Analyze GPU Roofline Design and Analyze Flow Graphs Minimize Analysis Overhead Analyze MPI Applications Manage Results Command Line Interface Troubleshooting Reference Appendix Notices and Disclaimers
Introduction x
Design and Optimization Methodology Tutorials and Samples Get Help and Support
Install and Launch Intel® Advisor x
Install Intel® Advisor Set Up Environment Variables Set Up System to Analyze GPU Kernels Set Up Environment to Offload SYCL, OpenMP* target, and OpenCL™ Applications to CPU Launch Intel® Advisor GUI Navigation Quick Start
Set Up Project x
Configure Target Application Build Target Application Create Project
Configure Target Application x
Limit the Number of Threads Used by Parallel Frameworks Choose a Small, Representative Data Set
Create Project x
Configure Project Configure Binary/Symbol Search Directories Configure Source Search Directory Binary/Symbol Search and Source Search Locations
Analyze Vectorization Perspective x
Run Vectorization and Code Insights Perspective from GUI Run Vectorization and Code Insights Perspective from Command Line Explore Vectorization and Code Insights Results
Run Vectorization and Code Insights Perspective from GUI x
Vectorization Accuracy Presets Customize Vectorization and Code Insights Perspective
Run Vectorization and Code Insights Perspective from Command Line x
Vectorization Accuracy Levels in Command Line
Explore Vectorization and Code Insights Results x
Vectorization Report Overview Examine Not-Vectorized and Under-Vectorized Loops Analyze Loop Call Count Investigate Memory Usage and Traffic Find Data Dependencies
Analyze CPU Roofline x
Run CPU / Memory Roofline Insights Perspective from GUI Run CPU / Memory Roofline Insights Perspective from Command Line Explore CPU/Memory Roofline Results
Run CPU / Memory Roofline Insights Perspective from GUI x
CPU Roofline Accuracy Presets Customize CPU / Memory Roofline Insights Perspective
Run CPU / Memory Roofline Insights Perspective from Command Line x
CPU Roofline Accuracy Levels in Command Line
Explore CPU/Memory Roofline Results x
CPU Roofline Report Overview Examine Bottlenecks on CPU Roofline Chart Examine Relationships Between Memory Levels Compare CPU Roofline Results
Model Threading Designs x
Run Threading Perspective from GUI Run Threading Perspective from Command Line Annotate Code for Deeper Analysis Explore Threading Results Model Threading Parallelism Check for Dependencies Issues Add Parallelism to Your Program
Run Threading Perspective from GUI x
Customize Threading Perspective
Run Threading Perspective from Command Line x
Threading Accuracy Levels in Command Line
Annotate Code for Deeper Analysis x
Annotate Code to Model Parallelism Annotations Annotation Report
Annotate Code to Model Parallelism x
Before Annotating Code for Deeper Analysis Use Amdahl's Law and Measure the Program Task Organization and Annotations Annotate Parallel Sites and Tasks Task Patterns Choose the Tasks Use Partially Parallel Programs with Intel® Advisor
Task Patterns x
Multiple Parallel Sites Data and Task Parallelism Mix and Match Tasks
Choose the Tasks x
Task Interactions and Suitability How Big Should a Task Be?
Annotations x
Annotation Types Annotation Definitions Files Add Annotations into Your Source Code Tips for Annotation Use with C/C++ Programs
Annotation Types x
Annotation Types Summary Annotation General Characteristics Site and Task Annotations for Simple Loops With One Task Site and Task Annotations for Parallel Sites with Multiple Tasks Lock Annotations Pause Collection and Resume Collection Annotations Special-purpose Annotations
Annotation Definitions Files x
Reference the Annotations Definitions Directory Add a Copy of the C/C++ Annotation Definition File to Your Visual Studio* Project Include the Annotations Header File in C/C++ Sources
Add Annotations into Your Source Code x
Insert Annotations Using the Annotation Wizard Copy Annotations and Build Settings Using the Annotation Assistant Pane Insert Annotations in the Visual Studio* Code Editor Insert Annotations in a Text Editor
Insert Annotations Using the Annotation Wizard x
Annotation Wizard - Page 1 Annotation Wizard - Page 2 Annotation Wizard - Page 3
Tips for Annotation Use with C/C++ Programs x
Control the Expansion of advisor-annotate.h Handle Compilation Issues that Appear After Adding advisor-annotate.h advisor-annotate.h and libittnotify.dll
Annotation Report x
Annotation Report, Clear Description of Storage Row Annotation Report, Disable Observations in Region Row Annotation Report, Pause Collection Row Annotation Report, Inductive Expression Row Annotation Report, Lock Row Annotation Report, Observe Uses Row Annotation Report, Reduction Row Annotation Report, Re-enable Observations at End of Region Row Annotation Report, Resume Collection Row Annotation Report, Site Row Annotation Report, Task Row Annotation Report, User Memory Allocator Use Row Annotation Report, User Memory Deallocator Use Row
Model Threading Parallelism x
Suitability Report Overview Choose Modeling Parameters in the Suitability Report Fix Annotation-related Errors Detected by the Suitability Tool Advanced Modeling Options Reduce Parallel Overhead, Lock Contention, and Enable Chunking
Reduce Parallel Overhead, Lock Contention, and Enable Chunking x
Reduce Site Overhead Reduce Task Overhead Reduce Lock Overhead Reduce Lock Contention Enable Task Chunking
Check for Dependencies Issues x
Code Locations Pane Filter Pane (Dependencies Report) Problems and Messages Pane Dependencies Source Window
Dependencies Source Window x
Code Locations Pane (Dependencies Source Window) Focus Code Location Pane Focus Code Location Call Stack Pane Related Code Locations Pane Related Code Location Call Stack Pane Relationship Diagram Pane
Add Parallelism to Your Program x
Before You Add Parallelism: Choose a Parallel Framework Add the Parallel Framework to Your Build Environment Annotation Report Replace Annotations with Intel® oneAPI Threading Building Blocks (oneTBB) Code Replace Annotations with OpenMP* Code Next Steps for the Parallel Program
Before You Add Parallelism: Choose a Parallel Framework x
Parallel Frameworks Intel® oneAPI Threading Building Blocks (oneTBB) OpenMP* Microsoft Task Parallel Library* (TPL) Other Parallel Frameworks
Add the Parallel Framework to Your Build Environment x
Enable Intel® oneAPI Threading Building Blocks (oneTBB) in your Build Environment Define the TBBROOT Environment Variable Enable C++11 Lambda Expression Support with Intel® oneAPI Threading Building Blocks (oneTBB) Enable OpenMP* in your Build Environment
Annotation Report x
Annotation Report Overview Locate Annotations with the Annotation Report
Replace Annotations with Intel® oneAPI Threading Building Blocks (oneTBB) Code x
Intel® oneAPI Threading Building Blocks (oneTBB) Mutexes Intel® oneAPI Threading Building Blocks (oneTBB) Simple Mutex - Example Test the Intel® oneAPI Threading Building Blocks (oneTBB) Synchronization Code Parallelize Functions - Intel® oneAPI Threading Building Blocks (oneTBB) Tasks Parallelize Data - Intel® oneAPI Threading Building Blocks (oneTBB) Counted Loops Parallelize Data - Intel® oneAPI Threading Building Blocks (oneTBB) Loops with Complex Iteration Control
Replace Annotations with OpenMP* Code x
Add OpenMP Code to Synchronize the Shared Resources OpenMP Critical Sections Basic OpenMP Atomic Operations Advanced OpenMP Atomic Operations OpenMP Reduction Operations OpenMP Locks Test the OpenMP Synchronization Code Parallelize Functions - OpenMP Tasks Parallelize Data - OpenMP Counted Loops Parallelize Data - OpenMP Loops with Complex Iteration Control
Next Steps for the Parallel Program x
Use Intel® Inspector and Intel® VTune™ Profiler Debug Parallel Programs
Model Offloading to a GPU x
Run Offload Modeling Perspective from GUI Run Offload Modeling Perspective from Command Line Explore Offload Modeling Results Advanced Modeling Configuration
Run Offload Modeling Perspective from GUI x
Offload Modeling Accuracy Presets Customize Offload Modeling Perspective
Run Offload Modeling Perspective from Command Line x
Offload Modeling Accuracy Levels in Command Line Run GPU-to-GPU Performance Modeling from Command Line
Explore Offload Modeling Results x
Offload Modeling Report Overview Examine Regions Recommended for Offloading Examine Data Transfers for Modeled Regions Check for Dependency Issues Explore Performance Gain from GPU-to-GPU Modeling Investigate Non-Offloaded Code Regions Cannot Be Modeled Less or Equally Profitable Than Children/Parent Offload Not Profitable N/A - Part of Offload Total Time Is Too Small for Reliable Modeling
Advanced Modeling Configuration x
Model Application Performance on a Custom Target GPU Device Check How Assumed Dependencies Affect Modeling Manage Invocation Taxes Enforce Offloading for Specific Loops
Analyze GPU Roofline x
Run GPU Roofline Insights Perspective from GUI Run GPU Roofline Insights Perspective from Command Line Explore GPU Roofline Results
Run GPU Roofline Insights Perspective from GUI x
GPU Roofline Accuracy Presets Customize GPU Roofline Insights Perspective
Run GPU Roofline Insights Perspective from Command Line x
GPU Roofline Accuracy Levels in Command Line
Explore GPU Roofline Results x
Examine GPU Roofline Summary Examine Bottlenecks on GPU Roofline Chart Examine Kernel Details Compare GPU Roofline Results
Design and Analyze Flow Graphs x
Where to Find the Flow Graph Analyzer Launching the Flow Graph Analyzer Flow Graph Analyzer GUI Overview Flow Graph Analyzer Workflows Designer Workflow Generating C++ Stubs Preferences Scalability Analysis Collecting Traces from Applications Nested Parallelism in Flow Graph Analyzer Analyzer Workflow Experimental Support for OpenMP* Applications Sample Trace Files Additional Resources
Flow Graph Analyzer GUI Overview x
Menus Toolbars Tabs Main Canvas Charts
Designer Workflow x
Adding Nodes, Edges, and Ports Modifying Node Properties Viewing Edge Properties Validating a Graph Saving a Graph to a File
Scalability Analysis x
Activating the Graph Scalability Analysis Prerequisites Running the Scalability Analysis Exploring the Parallelism in a Concurrent Node Showing Non-Parallel Nature of a Serial Node Explore Parallelism Provided by the Topology of a Graph Understanding Analysis Color Codes
Scalability Analysis Prerequisites x
Setting Concurrency Specification Setting Data Count Setting Node Weight
Collecting Traces from Applications x
Building an Application for Trace Collection Collecting Trace Files
Building an Application for Trace Collection x
Building an Application on Windows* OS Building an Application on Linux* OS Building an Application on macOS*
Collecting Trace Files x
Collect Traces In the Flow Graph Analyzer GUI Collect Traces Outside the Flow Graph Analyzer GUI
Collect Traces Outside the Flow Graph Analyzer GUI x
Collecting Trace Files with fgtrun Script Collecting Trace Files without fgtrun Script
Analyzer Workflow x
Find Time Regions of Low Concurrency and Their Cause Finding a Critical Path Finding Tasks with Small Durations Reduce Scheduler Overhead using Lightweight Policy Identifying Tasks that Operate on Common Input Support for SYCL
Support for SYCL x
Collect SYCL Application Traces Examine a SYCL Application Graph Find Issues Using Static Rule-check Engine
Examine a SYCL Application Graph x
Hotspot View View Performance Inefficiencies of Data-parallel Constructs
Find Issues Using Static Rule-check Engine x
Issue: Const Reference to a Host Pointer Used to Initialize a Buffer Issue: Host Pointer Accessor Used in a Loop Issue: Data Parallel Construct Inefficiency
Experimental Support for OpenMP* Applications x
Collecting Traces for OpenMP* Applications OpenMP* Constructs in the Per-Thread Task View OpenMP* Constructs in the Graph Canvas
Sample Trace Files x
code_generation Samples performance_analysis Samples
Minimize Analysis Overhead x
Collection Controls to Minimize Analysis Overhead Loop Markup to Minimize Analysis Overhead Filtering to Minimize Analysis Overhead Execution Speed/Duration/Scope Properties to Minimize Analysis Overhead Miscellaneous Techniques to Minimize Analysis Overhead
Analyze MPI Applications x
Model MPI Application Performance on GPU Control Collection with an MPI_Pcontrol Function
Manage Results x
Open a Result Rename an Existing Result Delete a Result Save Results to a Custom Location Work with Standalone HTML Reports Create a Read-only Result Snapshot Create a Result Snapshot Dialog Box Open a Result as a Read-only File in Visual Studio
Command Line Interface x
advisor Command Line Interface Reference Offload Modeling Command Line Reference Generate Pre-configured Command Lines
advisor Command Line Interface Reference x
advisor Command Action Reference advisor Command Option Reference
advisor Command Action Reference x
collect command create-project help import-dir mark-up-loops report snapshot version workflow
advisor Command Option Reference x
accuracy append app-working-dir assume-dependencies assume-hide-taxes assume-ndim-dependency assume-single-data-transfer auto-finalize batching benchmarks-sync bottom-up cache-binaries cache-binaries-mode cache-config cache-simulation cache-sources cachesim cachesim-associativity cachesim-cacheline-size cachesim-mode cachesim-sampling-factor cachesim-sets check-profitability clear config count-logical-instructions count-memory-instructions count-memory-objects-accesses count-mov-instructions count-send-latency cpu-scale-factor csv-delimiter custom-config data-limit data-reuse-analysis data-transfer data-transfer-histogram data-transfer-page-size data-type delete-tripcounts disable-fp64-math-optimization display-callstack dry-run duration dynamic enable-cache-simulation enable-data-transfer-analysis enable-task-chunking enforce-baseline-decomposition enforce-fallback enforce-offloads estimate-max-speedup evaluate-min-speedup exclude-files executable-of-interest exp-dir filter filter-by-scope filter-reductions flop force-32bit-arithmetics force-64bit-arithmetics format gpu gpu-carm gpu-kernel-of-interest gpu-sampling-interval hide-data-transfer-tax ignore ignore-app-mismatch ignore-checksums instance-of-interest integrated interval limit loop-call-count-limit loop-filter-threshold loops mark-up mark-up-list memory-level memory-operation-type mix mkl-user-mode model-baseline-gpu model-children model-extended-math model-system-calls module-filter module-filter-mode mpi-rank mrte-mode ndim-depth-limit option-file overlap-taxes pack profile-gpu profile-intel-perf-libs profile-jit profile-python profile-stripped-binaries project-dir quiet recalculate-time record-mem-allocations record-stack-frame reduce-lock-contention reduce-lock-overhead reduce-site-overhead reduce-task-overhead refinalize-survey remove report-output report-template result-dir resume-after return-app-exitcode search-dir search-n-dim select set-dependency set-parallel set-parameter show-all-columns show-all-rows show-functions show-loops show-not-executed show-report small-node-filter sort-asc sort-desc spill-analysis stack-access-granularity stack-stitching stack-unwind-limit stacks stackwalk-mode start-paused static-instruction-mix strategy support-multi-isa-binaries target-device target-gpu target-pid target-process target-system threading-model threads top-down trace-mode trace-mpi track-memory-objects track-stack-accesses track-stack-variables trip-counts verbose with-stack
Offload Modeling Command Line Reference x
run_oa.py Options collect.py Options analyze.py Options
Troubleshooting x
Error Message: Application Sets Its Own Handler for Signal Error Message: Cannot Collect GPU Hardware Metrics for the Selected GPU Adapter Error Message: Memory Model Cache Hierarchy Incompatible Error Message: No Annotations Found Error Message: No Data Is Collected Error Message: Stack Size Is Too Small Error Message: Undefined Linker References to dlopen or dlsym Problem: Broken Call Tree Problem: Code Region is not Marked Up Problem: Debug Information Not Available Problem: No Data Problem: Source Not Available Problem: Stack in the Top-Down Tree Window Is Incorrect Problem: Survey Tool does not Display Survey Report Problem: Unexpected C/C++ Compilation Errors After Adding Annotations Problem: Unexpected Unmatched Annotations in the Dependencies Report Warning: Analysis of Debug Build Warning: Analysis of Release Build
Reference x
Data Reference Dependencies Problem and Message Types Recommendation Reference User Interface Reference
Data Reference x
CPU Metrics Accelerator Metrics
Dependencies Problem and Message Types x
Dangling Lock Data Communication Data Communication, Child Task Inconsistent Lock Use Lock Hierarchy Violation Memory Reuse Memory Reuse, Child Task Memory Watch Missing End Site Missing End Task Missing Start Site Missing Start Task No Tasks in Parallel Site One Task Instance in Parallel Site Orphaned Task Parallel Site Information Thread Information Unhandled Application Exception
Recommendation Reference x
Vectorization Recommendations for C++ Vectorization Recommendations for Fortran
User Interface Reference x
Dialog Box: Corresponding Command Line Dialog Box: Create a Project Dialog Box: Create a Result Snapshot Dialog Box: Options - Assembly Editor Tab Dialog Box: Options - General Dialog Box: Options - Result Location Dialog Box: Project Properties - Analysis Target Dialog Box: Project Properties - Binary/Symbol Search Dialog Box: Project Properties - Source Search Pane: Advanced View Pane: Analysis Workflow Pane: Roofline Chart Pane: GPU Roofline Chart Project Navigator Pane Toolbar: Intel Advisor Annotation Report Window: Dependencies Source Window: GPU Roofline Regions Window: GPU Roofline Insights Summary Window: Memory Access Patterns Source Window: Offload Modeling Summary Window: Offload Modeling Report - Accelerated Regions Window: Perspective Selector Window: Refinement Reports Window: Suitability Report Window: Suitability Source Window: Survey Report Window: Survey Source Window: Threading Summary Window: Vectorization Summary
Window: Refinement Reports x
Tab: Dependencies Report Tab: Memory Access Patterns Report
Appendix x
Data Sharing Problems Notational Conventions Key Concepts Related Information
Data Sharing Problems x
Data Sharing Problem Types Problem Solving Strategies
Data Sharing Problem Types x
Incidental Sharing Independent Updates
Problem Solving Strategies x
Eliminate Incidental Sharing Synchronize Independent Updates Difficult Problems: Choosing a Different Set of Tasks Fix Problems in Code Used by Multiple Parallel Sites Memory That is Accessed Through a Pointer
Eliminate Incidental Sharing x
Examine the Task's Static and Dynamic Extent Verify Whether Incidental Sharing Exists Create the Private Memory Location Pointer Dereferences
Synchronize Independent Updates x
Synchronization Explicit Locking Assign Locks to Transactions Pitfalls from Using Synchronization
Key Concepts x
Glossary Parallelism
Parallelism x
Parallel Processing Terminology Add Parallelism Common Issues When Adding Parallelism Parallel Programming Implementations
Intel® Advisor User Guide

Investigate Non-Offloaded Code Regions

The modeling step analyzes code regions profitability for offloading to a target device. Some regions might be not profitable for offloading or cannot be modeled.

If you view a result in the Intel® Advisor GUI: To see details why your code region of interest is reported as not recommended for offloading, select a loop in a Code Regions pane and see the Details tab in the right-side pane for detailed loop information, including the reason why the loop is not recommended for offloading.

By default, the report shows all code regions. You can apply filters to see only regions recommended or not recommended for offloading: open the drop-down list and select the desired filter option.

If you view the result in an Offload Modeling HTML report: Go to the Non-Offloaded Regions tab and examine the Why Not Offloaded column in the Offload Information group to the reason why a code region is not recommended for offloading to a selected target platform.

TIP:
For each region not recommended for offloading, you can force offload modeling. See the Enforce Offloading for Specific Loops.

Cannot Be Modeled

Message

Cause and Details

Solution

Cannot be modeled: Outside of Marked Region

Intel® Advisor cannot model performance for a code region because it is not marked up for analysis.

Make sure a code region satisfies all markup rules or use a different markup strategy:

  • It is not a system module or a system function.
  • It has instruction mixes.
  • It is executed.
  • Its execution time is not less than 0.02 seconds.

Cannot be modeled: Not Executed

A code region is in the call tree, but the Intel Advisor detected no calls to it for a dataset used during Survey.

This can happen if execution time of the loop is very small and close to the sampling interval of the Intel Advisor. Such loops can have significant inaccuracies in time measurement. By default, the sampling interval for Survey is 0.01 seconds.

You can try to decrease the sampling interval of the Intel Advisor:

  • From GUI:
    1. Go to Project Properties > Survey Hotspots Analysis > Advanced.
    2. Set the Sampling Interval to less than 10ms.
    3. Re-run Offload Modeling.
  • From CLI: Use the --interval option when running the --collect=survey.

Cannot be modeled: Internal Error

Internal Error means incorrect data or lack of data because the Intel Advisor encountered issues when collecting or processing data.

Try to re-run the Offload Modeling perspective to fix the metrics attribution problem. If this does not help, use the Analyzers Community forum for technical support.

Cannot be modeled: System Module

This code region is a system function/loop.

This is not an issue. If this code region is inside an offload region, or a runtime call, its execution time is added to execution time of offloaded regions.

Cannot be modeled: No Execution Count

The Intel Advisor detected no calls to a loop during Trip Count step of the Characterization analysis and no information about execution counts is available for this loop.

Check what loop is executed at this branch.

If you see a wrong loop, try re-running the Offload Modeling to fix the metrics attribution problem.

Less or Equally Profitable Than Children/Parent Offload

This message is not an issue. It means that Intel Advisor has found a more profitable code region to offload. If you still want to see offload estimations for the original code region, use the solutions described in the table below.

Message

Cause and Details

Solution

Less or equally profitable than children offloads

Offloading child loops/functions of this code region is more profitable than offloading the whole region with all its children. This means that the execution Time estimated on a target platform for the region of interest is greater than or equal to the sum of Time values estimated on a target platform for its child regions profitable for offloading.

The following reasons might prevent offloading: total execution time, taxes, trip counts, dependencies.

Model offloading for specific code regions even if they are not profitable. See Enforce Offloading for Specific Loops for details.

Less or equally profitable than parent offload

Offloading a whole parent code region of the region of interest is more profitable than offloading any of its child regions separately. This means that the Time estimated on a target platform for the region of interest is greater than or equal to the Time estimated on the target platform for its parent region.

Offloading a child code region might be limited by high offload taxes.

Solution 1

If you assume the kernel execution should overlap offload taxes, use the --assume-hide-taxes option with --collect=projection action option or the analyze.py script. See Manage Invocation Taxes for details.

Solution 2

Model offloading for only specific code regions even if they are not profitable. See Enforce Offloading for Specific Loops for details.

Not Profitable

Message

Cause and Details

Solution

Not profitable: Parallel execution efficiency is limited due to Dependencies

Dependencies limit parallel execution and the code region cannot benefit from offloading to a target device. The estimated execution time after acceleration is greater than or equal to the original execution time.

Solution 1

Ignore assumed dependencies and model offloading for all or selected code regions:

  • From GUI:
    1. Go to Project Properties > Performance Modeling.

    2. Enter one of the options in the Other Parameters field:

      • --no-assume-dependencies to assume all code regions that do not have information about their dependency are parallel
      • --set-parallel=[<loop-IDs/source-locations>] to ignore dependencies for specified code regions
    3. Re-run Performance Modeling.
  • From CLI: When running --collect=projection or analyze.py, use one of the following:
    • --no-assume-dependencies to ignore dependencies for all code regions
    • --set-parallel=[<loop-IDs/source-locations>] to ignore dependencies for specified code regions

For details, see Check How Dependencies Affect Modeling.

Solution 2

If you did not enable the Dependencies analysis when collecting data, run the analysis as follows to get detailed information about real dependencies in your code:

  • From GUI: Enable the Dependencies and Performance Modeling analyses from the Analysis Workflow pane and re-run the perspective.
  • From CLI: Run the Dependencies analysis with --collect=dependencies and re-run the Performance Modeling with --collect=projection or analyze.py.

See Dependency Type metric description for details.

Not profitable: The Number of Loop Iterations is not enough to fully utilize Target Platform capabilities

The loop cannot benefit from offloading to a target platform as it has a low number of iterations.

In most cases, such code regions cannot benefit from offloading. If you assume that during code migration, the amount of parallel work grows and a loop is broken down into several chunks by a compiler or a program model, use the following workaround:

  • From GUI:
    1. Go to Project Properties > Performance Modeling.
    2. Enter --batching or --threads=<target-threads> in the Other Parameters field. <target-threads> is the number of parallel threads equal to the target device capacity.
    3. Re-run Performance Modeling.
  • From CLI: When running --collect=projection or analyze.py, use one of the following:
    • --batching to model batching-like techniques
    • --threads=<target-threads>, where <target-threads> is the number of parallel threads equal to the target device capacity

If you enable batching, the kernel invocation tax might grow. You can use the --assume-hide-taxes option to reduce the task. See Manage Invocation Taxes for details.

Not profitable: Data Transfer Tax is greater than Computation Time and Memory Bandwidth Time

Time spent on transferring data to a target device is greater than compute time and memory bandwidth time. The resulting time estimated on a target platform with data transfer tax is greater than or equal to the time measured on a host platform.

Check the Bounded By and Data Transfer Tax columns in the Estimated Bounded By column group and the Estimated Data Transfer with Reuse column group. Large value means that this code region cannot benefit from offloading.

See Bounded By for details about metric interpretation.

If you still want to offload such regions, disable data transfer analysis with the --data-transfer=off to use only estimated execution time for speedup and profitability calculation.

NOTE:
This option disables data transfer analysis for all loops. You might get different performance modeling results for all loops.

If you already collected data transfer metrics, you can turn off modeling data transfer tax with the command line option --hide-data-transfer-tax.

Not profitable: Computation Time is high despite the full use of Target Platform capabilities

The code region uses full target platform capabilities, but time spent for compute operations is still high. As a result, the execution time estimated on a target platform is greater than or equal to the time measured on a host platform.

Check the value in the Compute column in the Estimated Bound-by column group. Unexpectedly high value means one of the following:

  • There is a problem with a programming model used.
  • Target GPU compute capabilities are lower than baseline CPU compute capabilities.
  • Internal Intel Advisor error happened caused by incorrect compute time estimation.

Not profitable: Cache/Memory Bandwidth Time is greater than other execution time components on Target Device

The time spent in cache or memory bandwidth takes a big part of the time estimated on a target platform. As a result, it is greater than or equal to the time measured on a host platform.

In the report, the Cache/Memory is replaced with a specific cache or memory level that prevents offloading, for example, L3 or LLC. See the Throughput column for details about the highest bandwidth time.

  1. Examine code region children to identify which part takes most of the time and prevents offloading.
  2. Optimize the part of your code that takes most of the time measured on a baseline platform and rerun the perspective.

Not profitable because of offload overhead (taxes)

Total time of offload taxes, which includes Kernel Launch Tax, Data Transfer Tax, takes a big part of the time estimated on a target platform. As a result, it is greater than or equal to the time measured on a host platform.

Examine the Taxes with Reuse column in the Estimated Bounded by group for the biggest and total time taxes paid for offloading the code region to a target platform. Expand the Estimated Bounded by group to see a full picture of time taxes paid for offloading the region to the target platform. Big value in any of the columns means that this code region cannot benefit from offloading because the cost of offloading is high.

If kernel launch tax is large and you assume the kernel execution should overlap the launch tax, model hiding the launch taxes as follows:

  • From GUI: Enabled the Single Kernel Launch Tax option from the Analysis Workflow pane and rerun the Performance Modeling analysis.
  • From CLI: Use the --assume-hide-taxes option with the --collect=projection or analyze.py

See Manage Invocation Taxes for details.

Not profitable: Kernel Launch Tax is greater than Kernel Execution Time and Data Transfer Time

Time spent on launching a kernel is greater than execution time estimated on a target platform and estimated data transfer time. The resulting time estimated on the target platform with data transfer tax is greater than or equal to the time measured on a host platform.

Examine the Bounded By and Kernel Launch Tax columns in the Estimated Bounded By column group.

See Bounded By for details about metric interpretation.

High value in Kernel Launch Tax means that the Intel Advisor detects high call count for a potentially profitable code region and assumes that the kernel invocation tax is paid as many times as the kernel is launched. For this reason, it assumes that the code region cannot benefit from offloading.

If you assume the kernel execution should overlap the launch tax, model hiding the launch taxes as follows:

  • From GUI: Select the Single Kernel Launch Tax checkbox for the Performance Modeling analysis.
  • From CLI: Use the --assume-hide-taxes option with the --collect=projection action option or analyze.py.

For details, see Manage Invocation Taxes.

Not profitable: Atomic Throughput Time is greater than other execution time components on a Target Device

Atomic operations include loading, changing, and storing data to make sure it is not affected by other threads between the calls.

When modeling atomic operations, Intel Advisor assumes that all threads wait for each other, so Atomic Throughput time might be high and can be one of the main hotspots.

Go to the Analyzers Community forum for technical support and advice.

Not profitable: Instruction Latency is greater than Compute Time and Memory Bandwidth Time

Each memory read instruction produces a GPU thread stall. The stall is called a memory latency. Usually, execution of other threads can overlap it.

However, sometimes the amount of non-overlapped latency has a big impact on performance. Intel Advisor can estimate the non-overlapped memory latency and add it to the kernel estimated execution time.

If you reduce thread occupancy, it can increase the amount of non-overlapped memory latency

Examine the Latency column to see how much time spent for load latency and the Thread Occupancy column to understand the reason for this. Low occupancy means that this is the reason for a high load latency. In this case, when offloading the code, increase the kernel parallelism or cover latency with other instructions.

If you are sure that the load latency is overlapped with compute instructions in your code, you can enable latency hiding mode with the following:

  • From GUI:

    1. Go to Project Properties > Performance Modeling.

    2. Enter --count-send-latency=first in the Other Parameters field.

    3. Re-run Performance Modeling.
  • From CLI: Use the --count-send-latency=first option with the --collect=projection action option or analyze.py.

N/A - Part of Offload

This means that offloading a code region is less profitable than offloading its outer loop.

This is not an issue. The code region of interest is located inside of an offloaded loop.

Total Time Is Too Small for Reliable Modeling

This means the execution time of a code region or a whole loop nest is less than 0.02 seconds. In this case, Intel Advisor cannot estimate the speedup correctly and say if it is worth to offload the code regions because its execution time is close to the sampling interval of the Intel Advisor.

Possible Solution

If you want to check the profitability of offloading code regions with total time less than 0.02 seconds:

  • From GUI:
    1. Go to Project Properties > Performance Modeling.
    2. Enter the --loop-filter-threshold=0 option to the Other parameters field to model such small offloads.
    3. Re-run Performance Modeling.
  • From CLI: Use the --loop-filter-threshold=0 option with the --collect=projection or analyze.py.
Parent topic: Explore Offload Modeling Results
Level Two Title