Intel® MPI Library Developer Guide for Windows* OS

Download PDF
ID 768730
Date 11/07/2023
Public

A newer version of this document is available. Customers should click here to go to the newest version.

Document Table of Contents
Document Table of Contents x
Intel® MPI Library Developer Guide for Windows* OS
Intel® MPI Library Developer Guide for Windows* OS x
Introduction Installation and Prerequisites Compiling and Linking Running Applications Debugging Applications Analysis and Tuning Troubleshooting Notices and Disclaimers
Introduction x
Introducing Intel® MPI Library Conventions and Symbols Related Information
Installation and Prerequisites x
Installation Prerequisite Steps User Authorization
Compiling and Linking x
Compiling an MPI Program Configuring a Microsoft Visual Studio* Project
Running Applications x
Running Intel® MPI Library in Containers Selecting a Library Configuration Running an MPI Program Running an MPI/OpenMP* Program MPMD Launch Mode Fabrics Control Job Schedulers Support Controlling Process Placement Java* MPI Applications Support
Running Intel® MPI Library in Containers x
Build a Singularity* Container for an MPI Application Run the Application with a Container
Fabrics Control x
Selecting Fabrics Libfabric* Support OFI* Providers Support tcp netdir psm2 RxM
Debugging Applications x
Debugging Using -gtool for Debugging
Analysis and Tuning x
Displaying MPI Debug Information Tracing Applications MPI Tuning
Troubleshooting x
Error Message: Bad Termination Error Message: No such file or Directory Error Message: Permission Denied Error Message: Fatal Error Error Message: Bad File Descriptor Error Message: Too Many Open Files Problem: MPI Application Hangs Problem: Password Required Problem: Cannot Execute Binary File Problem: MPI limitation for Docker*
Intel® MPI Library Developer Guide for Windows* OS

OFI* Providers Support

Intel® MPI Library supports tcp, netdir, psm2,, verbs, and RxM OFI* providers. Each OFI provider is built as a separate dynamic library to ensure that a single libfabric* library can be run on top of different network adapters.

NOTE:
Use the environment variable FI_PROVIDER to select a provider. Set the FI_PROVIDER_PATH environment variable to specify the path to provider libraries.

To get a full list of environment variables available for configuring OFI, run the following command: 

> fi_info  -e

tcp

The TCP provider is a general purpose provider for the Intel MPI Library that can be used on any system that supports TCP sockets to implement the libfabric API. The provider lets you run the Intel MPI Library application over regular Ethernet, in a cloud environment that has no specific fast interconnect (e.g., GCP, Ethernet empowered Azure*, and AWS* instances) or using IPoIB.

The following runtime parameters can be used:

Name Description
FI_TCP_IFACE Specifies a particular network interface.
FI_TCP_PORT_LOW_RANGE

FI_TCP_PORT_HIGH_RANGE

 Sets the range of ports to be used by the TCP provider for its passive endpoint creation. This is useful when only a range of ports are allowed by the firewall for TCP connections.

netdir

The NETDIR provider runs over NetworkDirect* interface for the compatible hardware. The following runtime parameters can be used:

Name Description
FI_NETDIR_MR_CACHE Enables memory registration cache (default: 0)
FI_NETDIR_MR_CACHE_MAX_SIZE Sets the maximum memory registration cache size (default: mem/core_count/2)

psm2

The PSM2 provider runs over the PSM 2.x interface supported by the Intel® Omni-Path Fabric. PSM 2.x has all the PSM 1.x features, plus a set of new functions with enhanced capabilities. Since PSM 1.x and PSM 2.x are not application binary interface (ABI) compatible, the PSM2 provider works with PSM 2.x only and does not support Intel® True Scale Fabric.

The following runtime parameters can be used:

Name Description
FI_PSM2_INJECT_SIZE Define the maximum message size allowed for fi_inject and fi_tinject calls.  The default value is 64.
FI_PSM2_LAZY_CONN

Control the connection mode established between PSM2 endpoints that OFI endpoints are built on top of. When set to 0 (eager connection mode), connections are established when addresses are inserted into the address vector. When set to 1 (lazy connection mode), connections are established when addresses are used the first time in communication.

NOTE:
Lazy connection mode may reduce the start-up time on large systems at the expense of higher data path overhead.

RxM

The RxM (RDM over MSG) provider (ofi_rxm) is a utility provider that supports FI_EP_RDM endpoint emulated over FI_EP_MSG endpoint of the core provider.

The RxM provider requires the core provider to support the following features:

  • MSG endpoints (FI_EP_MSG)
  • FI_MSG transport (to support data transfers)
  • FI_RMA transport (to support rendezvous protocol for large messages and RMA transfers)
  • FI_OPT_CM_DATA_SIZE of at least 24 bytes

The following runtime parameters can be used:

Name Description
FI_OFI_RXM_BUFFER_SIZE Define the transmit buffer size/inject size. Messages of smaller size are transmitted via an eager protocol and those above would be transmitted via a rendezvous protocol. Transmitted data is copied up to the specified size. By default, the size is 16k.  
FI_OFI_RXM_SAR_LIMIT Сontrol the RxM SAR (Segmentation аnd Reassembly) protocol. Messages of greater size are transmitted via rendezvous protocol.
FI_OFI_RXM_USE_SRX

Control the RxM receive path. If the variable is set to 1, the RxM uses Shared Receive Context of the core provider. The default value is 0.

NOTE:
Setting this variable to 1 improves memory consumption, but may increase small message latency as a side-effect.
Parent topic: Fabrics Control
Level Two Title