Intel® MPI Library

Developer Reference for Linux* OS

Download PDF
ID 768732
Date 3/31/2025
Public
Document Table of Contents
Document Table of Contents x
Developer Reference for Linux* OS
Developer Reference for Linux* OS x
Introduction Command Reference Environment Variable Reference Miscellaneous Notices and Disclaimers
Introduction x
Introducing Intel® MPI Library Notational Conventions Related Information
Command Reference x
Compiler Commands mpirun mpiexec.hydra cpuinfo impi_info mpitune_fast
Compiler Commands x
Compilation Command Options
mpiexec.hydra x
Global Hydra Options Local Hydra Options gtool Options
Environment Variable Reference x
Compilation Environment Variables Hydra Environment Variables I_MPI_ADJUST Family Environment Variables Tuning Environment Variables Process Pinning Processor Identification Default Settings Pinning on Hybrid Architectures GPU Support NIC Pinning Environment Variables for Fabrics Control Environment Variables for Memory Policy Control Environment Variables for Asynchronous Progress Control Environment Variables for Multi-EP Other Environment Variables
Tuning Environment Variables x
Autotuning
Autotuning x
I_MPI_TUNING_AUTO Family Environment Variables
Process Pinning x
Processor Identification Default Settings Pinning on Hybrid Architectures Environment Variables for Process Pinning Interoperability with OpenMP* API
GPU Support x
GPU Pinning GPU Buffers Support
Environment Variables for Fabrics Control x
Communication Fabrics Control Shared Memory Control OFI*-capable Network Fabrics Control
Miscellaneous x
Java* Bindings for MPI-2 Routines
Developer Reference for Linux* OS

Process Pinning

Use this feature to pin a particular MPI process to a corresponding set of CPUs within a node and avoid undesired process migration. This feature is available on operating systems that provide the necessary kernel interfaces.

NOTE:
Starting with Intel(R) MPI 2021.15, the default topolib is ipl2. For legacy pinning, set I_MPI_HYDRA_TOPOLIB=hwloc. For more information, see Hydra Environment Variables.

Processor Identification

The following schemes are used to identify logical processors in a system:

  • System-defined logical enumeration
  • Topological enumeration based on three-level hierarchical identification through triplets (package/socket, core, thread)

The number of a logical CPU is defined as the corresponding position of this CPU bit in the kernel affinity bit-mask. Use the cpuinfo utility, provided with your Intel(R) MPI Library installation or the cat /proc/cpuinfo command to find out the logical CPU numbers.

The three-level hierarchical identification uses triplets that provide information about processor location and their order. The triplets are hierarchically ordered (package, core, and thread).

See the example for one possible processor numbering where there are two sockets, four cores (two cores per socket), and eight logical processors (two processors per core).

NOTE:
Logical and topological enumerations are not the same.
Logical Enumeration
0 4 1 5 2 6 3 7
Hierarchical Levels
Socket 0 0 0 0 1 1 1 1
Core 0 0 1 1 0 0 1 1
Thread 0 1 0 1 0 1 0 1
Topological Enumeration
0 1 2 3 4 5 6 7

Use the cpuinfo utility to identify the correspondence between the logical and topological enumerations. See Processor Information Utility for more details.

To calculate the pinning mask, use the Pinning Simulator for Intel MPI Library.

Default Settings

If you do not specify values for any process pinning environment variables, the default settings below are used. For details about these settings, see Environment Variables and Interoperability with OpenMP API.

  • I_MPI_HYDRA_TOPOLIB=ipl2
  • I_MPI_PIN=on
  • I_MPI_PIN_RESPECT_CPUSET=on
  • I_MPI_PIN_CELL=unit
  • I_MPI_PIN_ORDER=bunch

When the number of ranks per package can be evenly distributed to NUMAs within a package, the NUMA aware bunch algorithm is enabled.

Pinning on Hybrid Architectures

For Intel(R) CPUs with performance and efficient cores, the default is I_MPI_PIN=0, and each process inherits a pinning mask from the operating system. In such cases, pinning environment variables have no effect.

To use pinning environment variables, set I_MPI_PIN=1. In this case, the pinning library runs the default algorithm treating all cores the same. As a result, some MPI ranks could get more efficiency core than the others that may impact performance.

To remove efficient or performance cores from pinning, use I_MPI_PIN_PROCESSOR_EXCLUDE_LIST.

Parent topic: Environment Variable Reference
Level Two Title