Intel® MPI Library Developer Guide for Linux* OS

Download PDF
ID 768728
Date 12/16/2022
Public

A newer version of this document is available. Customers should click here to go to the newest version.

Document Table of Contents
Document Table of Contents x
Intel® MPI Library Developer Guide for Linux* OS
Intel® MPI Library Developer Guide for Linux* OS x
Introduction Installation and Prerequisites Compiling and Linking Running Applications Debugging Applications Analysis and Tuning Troubleshooting Additional Supported Features Examples Notices and Disclaimers
Introduction x
Introducing Intel® MPI Library Conventions and Symbols Related Information
Installation and Prerequisites x
Installation Prerequisite Steps
Compiling and Linking x
Compiling an MPI Program Compilers Support ILP64 Support
Running Applications x
Running Intel® MPI Library in Containers Selecting a Library Configuration Running an MPI Program Running an MPI/OpenMP* Program MPMD Launch Mode Fabrics Control Job Schedulers Support Controlling Process Placement Java* MPI Applications Support
Running Intel® MPI Library in Containers x
Build a Singularity* Container for an MPI Application Run the Application with a Container
Fabrics Control x
Selecting Fabrics Libfabric* Support OFI* Providers Support
Debugging Applications x
Debugging Using -gtool for Debugging
Analysis and Tuning x
Displaying MPI Debug Information Tracing Applications Interoperability with Other Tools Through -gtool MPI Tuning
Troubleshooting x
Error Message: Bad Termination Error Message: No such file or Directory Error Message: Permission Denied Error Message: Fatal Error Error Message: Bad File Descriptor Error Message: Too Many Open Files Problem: High Memory Consumption Readings Problem: MPI Application Hangs Problem: Password Required Problem: Cannot Execute Binary File Problem: MPI limitation for Docker*
Additional Supported Features x
Asynchronous Progress Control Multiple Endpoints Support
Multiple Endpoints Support x
MPI_THREAD_SPLIT Programming Model Threading Runtimes Support Program Examples Code Change Guide
Examples x
async_progress_sample.c thread_split.cpp thread_split_omp_for.c thread_split_omp_task.c thread_split_pthreads.c
Intel® MPI Library Developer Guide for Linux* OS

Troubleshooting

This section provides the troubleshooting information on typical MPI failures with corresponding output messages and behavior when a failure occurs.

If you encounter errors or failures when using the Intel® MPI Library, take the following general troubleshooting steps first:

  1. Check the Intel® MPI Library System Requirements section and the Known Issues section in the Intel® MPI Library Release Notes.
  2. Check accessibility of the hosts. Run a simple non-MPI application (for example, the hostname utility) on the problem hosts using mpirun. For example: 
    $ mpirun -ppn 1 -n 2 -hosts node01,node02 hostname  
node01 
node02

    This may help reveal an environmental problem (such as the MPI remote access mechanism is not configured properly), or a connectivity problem (such as unreachable hosts).

  3. Run the MPI application with debug information enabled: set the environment variables I_MPI_DEBUG=6 and/or I_MPI_HYDRA_DEBUG=on. Increase the integer value of debug level to get more information. This action helps narrow down to the problematic component.
  4. If you have the availability, download and install the latest version of Intel MPI Library from the official product page and check if your problem persists.
  5. If the problem still persists, you can submit a ticket via the Support page, or ask experts on the community forum.
Parent topic: Intel® MPI Library Developer Guide for Linux* OS
Level Two Title