Intel® Inspector User Guide for Linux* OS

Download PDF
ID 767796
Date 7/13/2023
Public

A newer version of this document is available. Customers should click here to go to the newest version.

Document Table of Contents
Document Table of Contents x
Intel® Inspector User Guide for Linux* OS
Intel® Inspector User Guide for Linux* OS x
Get Started Before You Begin Set Up a Project Collect Results Investigate Results Test for Regressions Use Cases Suppressions Support API Support Troubleshooting User Interface Reference Problem Type Reference Command Line Interface Support MPI Applications Support Appendix Notices and Disclaimers
Get Started x
Dynamic Analysis vs. Static Analysis Key Terminology Sample Applications and Tutorials Workflows-Quick Paths to Maximizing Productivity
Workflows-Quick Paths to Maximizing Productivity x
Analysis Workflow Reporting Workflow Regression Testing Workflow
Before You Begin x
Intel Inspector GUI Startup Tasks
Startup Tasks x
Change Result and Result Directory Name Templates Choose Baseline Results for Setting States Choose Debuggers Display Application Output
Set Up a Project x
Build Your Application Choose Projects Configure Projects
Choose Projects x
Choose Projects in the Intel Inspector GUI Create Projects in the Intel Inspector GUI
Configure Projects x
Binary/Symbol Search and Source Search Locations Choose Data Sets Manage Suppression Rules Search Non-standard Directories Set Project Properties-Advanced Set Target Application Properties - Basic
Collect Results x
Configure Analysis Run Analysis Examine Result Data During Analysis
Configure Analysis x
Memory Error Analysis Types Threading Error Analysis Types Manage Custom Analysis Types
Run Analysis x
Run Memory Error and Threading Error Analyses Stop Analysis
Examine Result Data During Analysis x
Find Memory Leaks On Demand Measure Memory Growth
Investigate Results x
View Results Choose Problems Interpret Result Data and Resolve Issues Investigate Problems Using Interactive Debugging Collaborate on Results Export and Import Files Compare Results
View Results x
Manage Help Snippet Displays Open Results Show Result Summaries
Choose Problems x
Deselecting Filter Criteria Refining the Source File Set Selecting Filter Criteria Selecting Problems
Interpret Result Data and Resolve Issues x
Severity Levels States Access the Explain Problem Help Change States Customize Frame Presentation Customize Data Columns Edit Source Code
Investigate Problems Using Interactive Debugging x
Extended Debugging Commands Stop Analyses During Interactive Debugging
Collaborate on Results x
State Merge Merge States Across Results Report Result Data
Export and Import Files x
Export Archives Import Archives and Other Files
Test for Regressions x
Tes for Regressions: Recommended Approach Test for Regressions: Other Approaches
Use Cases x
Analyze Offloaded Code
Analyze Offloaded Code x
Example: Memory Problem Example: Threading Problem
Suppressions Support x
Suppressions Support Using the GUI Suppressions Support Using Handwritten Suppression Rules Suppressions Support Using Third-party Suppression Rules
Suppressions Support Using the GUI x
Typical Suppressions Usage Model Using the GUI Suppression Rule Reach in the GUI Suppression Rule Examples in the GUI Defining Suppression Rules in the GUI Deleting Suppression Rules Applied to Marked Result Data in the GUI
Suppressions Support Using Handwritten Suppression Rules x
Typical Suppressions Usage Models Using Handwritten Suppression Files Suppression Rule Syntax in Text Format Suppression Rule Examples in Text Format
Suppressions Support Using Third-party Suppression Rules x
Typical Suppressions Usage Models Using Third-party Suppression Files Third-party Suppression Files
API Support x
APIs for Custom Synchronization APIs for Custom Memory Allocation APIs for Collection Control
Troubleshooting x
Troubleshooting Anti-virus Software Issues Troubleshooting Application Crashes Troubleshooting Internal Thread Suspension Attempt Troubleshooting No Problems Detected Troubleshooting No Symbolic Information Troubleshooting OpenMP* Technology Issues Troubleshooting Out-of-memory Conditions
User Interface Reference x
Context Menus: Project Navigator Context Menus: Sources Window Panes Context Menus: Summary Window Panes Dialog Box: Corresponding inspxe-cl Command Options Dialog Box: Create a Project Dialog Box: Create Suppression Dialog Box: Custom Analysis Dialog Box: Delete Suppressions Dialog Box: Export Result Dialog Box: Merge States Dialog Box: Options-Debugger Dialog Box: Options-General Dialog Box: Options-Result Location Dialog Box: Options-State Management Dialog Box: Problem Report Dialog Box: Project Properties-Binary/Symbol Search Dialog Box: Project Properties-Source Search Dialog Box: Project Properties-Suppressions Dialog Box: Project Properties-Target Dialog Box: Refine Source File Set Dialog Box: Select Stack Frame(s) Dialog Box: View Stack Hot Keys Pane: Analysis Type-Custom Pane: Analysis Type-Memory Errors Pane: Analysis Type-Threading Errors Pane: Application Output Pane: Code and Stack Pane: Code Locations Pane: Collection Log Pane: Collector Messages Pane: Compare Results Pane: Filters Pane: Import Result Pane: Launch Application Pane: Problems Pane: Project Navigator Pane: Timeline Toolbar: Command Toolbar: Intel Inspector Toolbar: Navigation Window: Collection Log Window: Compare Results Window: Import Result Window: Sources Window: Summary After Analysis Is Complete Window: Summary During Analysis
Problem Type Reference x
Asynchronous Buffer Cross-thread Stack Access Data Race Deadlock Host Pointer Used on Device Incorrect memcpy Call Invalid Deallocation Invalid Kernel Argument Invalid Kernel Argument Size Invalid Memory Access Invalid Partial Memory Access Lock Hierarchy Violation Memory Growth Memory Leak Memory Not Deallocated Mismatched Allocation/Deallocation Mismatched Queue Missing Allocation Non-Host Pointer Pointer from Different Device Thread Exit Information Thread Start Information Unhandled Application Exception Uninitialized Memory Access Uninitialized Partial Memory Access
Command Line Interface Support x
Command Syntax Command Syntax Alternatives and Rules Command Line Output Collecting Result Data from the Command Line Working with Reports from the Command Line Working with Suppressions from the Command Line inspxe-cl Actions, Options and Arguments
Working with Reports from the Command Line x
Reporting Result Data from the Command Line Saving and Formatting Reports from the Command Line Interpreting Result Data from the Command Line
Working with Suppressions from the Command Line x
Creating a Suppress-All File from the Command Line Converting Third-Party Suppression Files from the Command Line Applying Suppression Files from the Command Line Workaround for Disabled Suppressions from the Command Line
inspxe-cl Actions, Options and Arguments x
appdebug app-working-dir archive-name baseline-result collect collect-with command convert-suppression-file create-breakpoints create-suppression-file csv-delimiter debug-this executable-of-interest export filter finalize format help include-snippets include-sources import knob knob-list merge-states module-filter module-filter-mode no-auto-finalize no-summary option-file quiet report report-all report-output result-dir return-app-exitcode search-dir sort-asc sort-desc suppression-file user-data-dir verbose version
MPI Applications Support x
MPI Analysis Workflow Configure Installation Options Collect MPI Performance/Correctness Data Example See Also Finalize MPI Collected Data View MPI Collected Data MPI Analysis Limitations Support of Non-Intel MPI Implementations Additional MPI Resources
Appendix x
Product Design Sample Code Caveats Intel Inspector Filenames and Locations Notational Conventions Related Information System APIs Supported During Memory Error Analysis System APIs Supported During Threading Error Analysis Evaluation Features
Intel® Inspector User Guide for Linux* OS

Collect MPI Performance/Correctness Data

To collect performance or correctness data for an MPI application with the Intel® VTune™ Profiler / Intel Inspector on a Windows* or Linux* OS, the following command should be used:

$ mpirun-n <N> <abbr>-cl -r my_result -collect <analysis type> my_app [my_app_ options]

where <abbr> is amplxe or inspxe respectively. The list of analysis types available can be viewed using amplxe-cl-help collect command.

As a result of using the collection commands, a number of result directories are created in the current directory, named as my_result.0 - my_result.3. The numeric suffix is the corresponding MPI process rank that is detected and captured by the collector automatically. The usage of the suffix makes sure that multiple amplxe-cl / inspxe-cl instances launched in the same directory on different nodes do not overwrite the data of each other and can work in parallel. So, a separate result directory is created for each analyzed process in the job.

Sometimes it is necessary to collect data for a subset of the MPI processes in the workload. In this case the per-host syntax of mpirun/mpiexec* should be used to specify different command lines to execute for different processes.

When launching the collection on Windows OS, we recommend passing the -genvall option to the mpiexec tool to make sure that the user environment variables are passed to all instances of the profiled process. Otherwise, by default the processes are launched in the context of a system account and some environment variables (USERPROFILE, APPDATA) do not point where the tools expect them to point to.

There are also some specialties about stdout / stdin behavior in MPI jobs profiled with the tools:

  • It is recommended to pass the -quiet / -q option to amplxe-cl / inspxe-cl to avoid diagnostic output like progress messages being spilled to the console by every tool process in the job.

  • The user may want to use the -l option for mpiexec/mpirun to get stdout lines marked with MPI rank.

Example

The most reasonable analysis type to start with for the Intel VTune Profiler is hotspots, so an example of full command line for collection would be:

$ mpirun-n 4 amplxe-cl -r my_result -collect hotspots -- my_app [my_app_ options]

A similar command line for the Intel Inspector and its ti1/mi1 analysis types (the lowest overhead threading and memory correctness analysis types respectively) would look like:

$ mpirun-n 4 inspxe-cl -r my_result -collect mi1 -- my_app [my_app_ options]

$ mpirun-n 4 inspxe-cl -r my_result -collect ti1 -- my_app [my_app_ options]

Here is an example where there are 16 processes in the job distributed across the hosts and hotspots data should be collected for only two of them:

$ mpirun-host myhost -n 14 ./a.out : -host myhost -n 2 amplxe-cl -r foo -c hotspots ./a.out

As a result, two directories will be created in the current directory: foo.14 and foo.15 (given that process ranks 14 and 15 were assigned to the last 2 processes in the job). As an alternative to specifying the command line above, it is possible to create a configuration file with the following content: 

# config.txt configuration file
-host myhost -n 14 ./a.out
-host myhost -n 2 amplxe-cl -quiet -collect hotspots -r foo ./a.out

and run the data collection as:

$ mpirun-configfile ./config.txt

to achieve the same result as above (foo.14 and foo.15 result directories will be created). Similarly, you can use specific host names to control where the analyzed processes are executed: 

# config.txt configuration file
-host myhost1 -n 14 ./a.out
-host myhost2 -n 2 amplxe-cl -quiet -collect hotspots -r foo ./a.out

When the host names are mentioned, consecutive MPI ranks are allocated to the specified hosts. In the case above, ranks 0 to 13, inclusive, will be assigned to myhost1, the remaining ranks 14 and 15 will be assigned to myhost2. On Linux, it is possible to omit specifying the exact hosts, in which case the distribution of the processes between the hosts will be done in round-robin fashion. That is, myhost1 will get MPI ranks 0, 2, and 4 thru 15, while myhost2 will get MPI ranks 1 and 3. The latter behavior may change in the future.

NOTE:

In the examples this reference uses the mpirun command as opposed to mpiexec and mpiexec.hydra while real-world jobs might use the mpiexec* ones. mpirun is a higher-level command that dispatches to mpiexec or mpiexec.hydra depending on the current default and options passed. All the examples listed in the paper work for the mpiexec* commands as well as the mpirun command.

Parent topic: MPI Applications Support
Level Two Title