Intel® MPI Library Developer Guide for Linux* OS

Download PDF
ID 768728
Date 3/22/2024
Public

A newer version of this document is available. Customers should click here to go to the newest version.

Document Table of Contents
Document Table of Contents x
Intel® MPI Library Developer Guide for Linux* OS
Intel® MPI Library Developer Guide for Linux* OS x
Introduction Installation and Prerequisites Compiling and Linking Running Applications Debugging Applications Analysis and Tuning Troubleshooting Additional Supported Features Examples Notices and Disclaimers
Introduction x
Introducing Intel® MPI Library Conventions and Symbols Related Information
Installation and Prerequisites x
Installation Prerequisite Steps
Compiling and Linking x
Compiling an MPI Program Compilers Support ILP64 Support
Running Applications x
Running Intel® MPI Library in Containers Selecting a Library Configuration Running an MPI Program Running an MPI/OpenMP* Program MPMD Launch Mode Fabrics Control Job Schedulers Support Controlling Process Placement Java* MPI Applications Support
Running Intel® MPI Library in Containers x
Build a Singularity* Container for an MPI Application Run the Application with a Container
Fabrics Control x
Selecting Fabrics Libfabric* Support OFI* Providers Support
Debugging Applications x
Debugging Using -gtool for Debugging
Analysis and Tuning x
Displaying MPI Debug Information Tracing Applications Interoperability with Other Tools Through -gtool Using -gtool for Debugging See Also MPI Tuning
Troubleshooting x
Error Message: Bad Termination Error Message: No such file or Directory Error Message: Permission Denied Error Message: Fatal Error Error Message: Bad File Descriptor Error Message: Too Many Open Files Problem: High Memory Consumption Readings Problem: MPI Application Hangs Problem: Password Required Problem: Cannot Execute Binary File Problem: MPI limitation for Docker*
Additional Supported Features x
Asynchronous Progress Control Multiple Endpoints Support
Multiple Endpoints Support x
MPI_THREAD_SPLIT Programming Model Threading Runtimes Support Program Examples Code Change Guide
Examples x
async_progress_sample.c thread_split.cpp thread_split_omp_for.c thread_split_omp_task.c thread_split_pthreads.c
Intel® MPI Library Developer Guide for Linux* OS

Interoperability with Other Tools Through -gtool

To simplify interoperability with other analysis tools, Intel® MPI Library provides the -gtool option (also available as the I_MPI_GTOOL environment variable). Using the -gtool option, you can analyze specific MPI processes with VTune™ Profiler, Intel® Advisor, Valgrind,* and other tools through the mpiexec.hydra or mpirun commands.

The Intel Advisor CLI uses -gtool to analyze MPI applications. For sample command lines and example scenarios, including the Weather Research and Forecasting (WRF) Model, see Analyze Vectorization and Memory Aspects of an MPI Application.

If you are not using the -gtool option, to analyze an MPI process with, for example, VTune Profiler, you must specify the relevant command in the corresponding argument set: 

$ mpirun -n 3 ./myprog : -n 1 vtune -c advanced-hotspots -r ah -- ./myprog

The -gtool option allows you to specify a single analysis command for all argument sets (separated by colons ':') at once. Even though it is allowed to use -gtool within a single argument set, it is not recommended to use it in several sets at once and combine the two analysis methods (with -gtool and argument sets).

For example, to analyze processes 3, 5, 6, and 7 with the VTune Profiler, you can use the following command line: 

$ mpirun -n 8 -gtool "vtune -collect hotspots -r result:3,5-7" ./myprog

The -gtool option also provides several methods for finer process selection. For example, you can easily analyze only one process on each host, using the exclusive launch mode: 

$ mpirun -n 8 -ppn 4 -hosts node1,node2 -gtool "vtune -collect hotspots -r result:all=exclusive" ./myprog

You can also use the -gtoolfile option to specify -gtool parameters in a configuration file. All the same rules apply. Additionally, you can separate different command lines with section breaks.

For example, if gtool_config_file contains the following settings: 

env VARIABLE1=value1 VARIABLE2=value2:3,5,7-9; env VARIABLE3=value3:0,11
env VARIABLE4=value4:1,12

The following command sets VARIABLE1 and VARIABLE2 for processes 3, 5, 7, 8, and 9 and sets VARIABLE3 for processes 0 and 11, while VARIABLE4 is set for processes 1 and 12: 

$ mpirun -n 16 -gtoolfile gtool_config_file a.out

Using -gtool for Debugging

The -gtool runtime option can help you with debugging, when attaching to several processes at once. Instead of attaching to each process individually, you can specify all the processes in a single command line. For example: 

$ mpirun -n 16 -gtool "gdb:3,5,7-9=attach" ./myprog

The command line above attaches the GNU* Debugger (GDB*) to processes 3, 5, 7, 8 and 9.

NOTE:
Do not use the -gdb and -gtool options together. Use one option at a time.

Parent topic: Analysis and Tuning
Level Two Title