Developer Guide

Developer Guide for Intel® oneAPI Math Kernel Library Linux*

Download PDF
ID 766690
Date 12/16/2022
Public

A newer version of this document is available. Customers should click here to go to the newest version.

Document Table of Contents
Document Table of Contents x
Developer Guide for Intel® oneAPI Math Kernel Library for Linux*
Developer Guide for Intel® oneAPI Math Kernel Library for Linux* x
Getting Help and Support What's New Notational Conventions Related Information Getting Started Structure of the Intel® oneAPI Math Kernel Library Linking Your Application with the Intel® oneAPI Math Kernel Library Managing Performance and Memory Language-specific Usage Options Obtaining Numerically Reproducible Results Coding Tips Managing Output Working with the Intel® oneAPI Math Kernel Library Cluster Software Managing Behavior of the Intel® oneAPI Math Kernel Library with Environment Variables Configuring Your Integrated Development Environment to Link with Intel® oneAPI Math Kernel Library Intel® oneAPI Math Kernel Library Benchmarks Appendix A: Intel® oneAPI Math Kernel Library Language Interfaces Support Appendix B: Support for Third-Party Interfaces Appendix C: Directory Structure in Detail Notices and Disclaimers
Getting Started x
Shared Library Versioning CMake Config for oneMKL Checking Your Installation Setting Environment Variables Compiler Support Using Code Examples What You Need to Know Before You Begin Using the Intel® oneAPI Math Kernel Library
Setting Environment Variables x
Scripts to Set Environment Variables Modulefiles to Set Environment Variables Automating the Process of Setting Environment Variables Using the CMake Config File
Structure of the Intel® oneAPI Math Kernel Library x
Architecture Support High-level Directory Structure Layered Model Concept
Linking Your Application with the Intel® oneAPI Math Kernel Library x
Linking Quick Start Linking Examples Linking in Detail Building Custom Shared Objects
Linking Quick Start x
Using the -qmkl Compiler Option Using the Single Dynamic Library Selecting Libraries to Link with Using the Link-line Advisor Using the Command-Line Link Tool
Linking Examples x
Linking on IA-32 Architecture Systems Linking on Intel(R) 64 Architecture Systems
Linking in Detail x
Listing Libraries on a Link Line Dynamically Selecting the Interface and Threading Layer Linking with Interface Libraries Linking with Threading Libraries Linking with Computational Libraries Linking with Compiler Run-time Libraries Linking with System Libraries
Linking with Interface Libraries x
Using the ILP64 Interface vs. LP64 Interface Linking with Fortran 95 Interface Libraries
Building Custom Shared Objects x
Using the Custom Shared Object Builder Composing a List of Functions Specifying Function Names Distributing Your Custom Shared Object
Managing Performance and Memory x
Improving Performance with Threading Improving Performance for Small Size Problems Other Tips and Techniques to Improve Performance Using Memory Functions
Improving Performance with Threading x
OpenMP* Threaded Functions and Problems Functions Threaded with Intel® Threading Building Blocks Avoiding Conflicts in the Execution Environment Techniques to Set the Number of Threads Setting the Number of Threads Using an OpenMP* Environment Variable Changing the Number of OpenMP* Threads at Run Time Using Additional Threading Control Calling oneMKL Functions from Multi-threaded Applications Using Intel® Hyper-Threading Technology Managing Multi-core Performance Managing Performance with Heterogeneous Cores
Using Additional Threading Control x
oneMKL-specific Environment Variables for OpenMP Threading Control MKL_DYNAMIC MKL_DOMAIN_NUM_THREADS <span class='keyword'>MKL_NUM_STRIPES</span> Setting the Environment Variables for Threading Control
Improving Performance for Small Size Problems x
Using MKL_DIRECT_CALL in C Applications Using MKL_DIRECT_CALL in Fortran Applications Limitations of the Direct Call
Other Tips and Techniques to Improve Performance x
Coding Techniques Improving oneMKL Performance on Specific Processors Operating on Denormals
Using Memory Functions x
Avoiding Memory Leaks in oneMKL Using High-bandwidth Memory with oneMKL Redefining Memory Functions
Language-specific Usage Options x
Using Language-Specific Interfaces with Intel® oneAPI Math Kernel Library Mixed-language Programming with the Intel Math Kernel Library
Using Language-Specific Interfaces with Intel® oneAPI Math Kernel Library x
Interface Libraries and Modules Fortran 95 Interfaces to LAPACK and BLAS Compiler-dependent Functions and Fortran 90 Modules
Mixed-language Programming with the Intel Math Kernel Library x
Calling LAPACK, BLAS, and CBLAS Routines from C/C++ Language Environments Using Complex Types in C/C++ Calling BLAS Functions that Return the Complex Values in C/C++ Code
Obtaining Numerically Reproducible Results x
Getting Started with Conditional Numerical Reproducibility Specifying Code Branches Reproducibility Conditions Setting the Environment Variable for Conditional Numerical Reproducibility Code Examples
Coding Tips x
Example of Data Alignment Using Predefined Preprocessor Symbols for Intel® MKL Version-Dependent Compilation
Managing Output x
Using oneMKL Verbose Mode
Using oneMKL Verbose Mode x
Version Information Line Call Description Line
Working with the Intel® oneAPI Math Kernel Library Cluster Software x
Linking with oneMKL Cluster Software Setting the Number of OpenMP* Threads Using Shared Libraries Setting Environment Variables on a Cluster Interaction with the Message-passing Interface Using a Custom Message-Passing Interface Examples of Linking for Clusters
Examples of Linking for Clusters x
Examples for Linking a C Application Examples for Linking a Fortran Application
Managing Behavior of the Intel® oneAPI Math Kernel Library with Environment Variables x
Managing Behavior of Function Domains with Environment Variables Instruction Set Specific Dispatching on Intel® Architectures
Managing Behavior of Function Domains with Environment Variables x
Setting the Default Mode of Vector Math with an Environment Variable Managing Performance of the Cluster Fourier Transform Functions Managing Invalid Input Checking in LAPACKE Functions
Configuring Your Integrated Development Environment to Link with Intel® oneAPI Math Kernel Library x
Configuring the Eclipse* IDE CDT to Link with oneMKL
Intel® oneAPI Math Kernel Library Benchmarks x
Intel Optimized LINPACK Benchmark for Linux* Intel® Distribution for LINPACK* Benchmark Intel® Optimized High Performance Conjugate Gradient Benchmark
Intel Optimized LINPACK Benchmark for Linux* x
Contents of the Intel® Optimized LINPACK Benchmark Running the Software Known Limitations of the Intel® Optimized LINPACK Benchmark
Intel® Distribution for LINPACK* Benchmark x
Overview of the Intel® Distribution for LINPACK* Benchmark Contents of the Intel® Distribution for LINPACK* Benchmark Building the Intel® Distribution for LINPACK* Benchmark for a Customized MPI Implementation Building the Netlib HPL from Source Code Configuring Parameters Ease-of-use Command-line Parameters Running the Intel® Distribution for LINPACK* Benchmark Heterogeneous Support in the Intel® Distribution for LINPACK* Benchmark Environment Variables Improving Performance of Your Cluster
Intel® Optimized High Performance Conjugate Gradient Benchmark x
Overview of the Intel Optimized HPCG Versions of the Intel Optimized HPCG Getting Started with Intel Optimized HPCG Choosing Best Configuration and Problem Sizes
Appendix A: Intel® oneAPI Math Kernel Library Language Interfaces Support x
Language Interfaces Support, by Function Domain Include Files
Appendix B: Support for Third-Party Interfaces x
FFTW Interface Support
Appendix C: Directory Structure in Detail x
Detailed Structure of the IA-32 Architecture Directories Detailed Structure of the Intel® 64 Architecture Directories
Detailed Structure of the IA-32 Architecture Directories x
Static Libraries in the <span class='filepath'>lib</span> <span class='filepath'>/</span> <span class='filepath'>ia32</span> <span class='filepath'>_lin</span> Directory Dynamic Libraries in the <span class='filepath'>lib</span> <span class='filepath'>/</span> <span class='filepath'>ia32</span> <span class='filepath'>_lin</span> Directory
Detailed Structure of the Intel® 64 Architecture Directories x
Static Libraries in the <span class='filepath'>lib</span> <span class='filepath'>/</span> <span class='filepath'>intel64</span> <span class='filepath'>_lin</span> Directory Dynamic Libraries in the <span class='filepath'>lib</span> <span class='filepath'>/</span> <span class='filepath'>intel64</span> <span class='filepath'>_lin</span> Directory
Developer Guide for Intel® oneAPI Math Kernel Library for Linux*

MKL_DOMAIN_NUM_THREADS

The MKL_DOMAIN_NUM_THREADS environment variable suggests the number of OpenMP threads for a particular function domain.

MKL_DOMAIN_NUM_THREADS accepts a string value <MKL-env-string>, which must have the following format:

<MKL-env-string> ::= <MKL-domain-env-string> { <delimiter><MKL-domain-env-string> }

<delimiter> ::= [ <space-symbol>* ] ( <space-symbol> | <comma-symbol> | <semicolon-symbol> | <colon-symbol>) [ <space-symbol>* ]

<MKL-domain-env-string> ::= <MKL-domain-env-name><uses><number-of-threads>

<MKL-domain-env-name> ::= MKL_DOMAIN_ALL | MKL_DOMAIN_BLAS | MKL_DOMAIN_FFT | MKL_DOMAIN_VML | MKL_DOMAIN_PARDISO

<uses> ::= [ <space-symbol>* ] ( <space-symbol> | <equality-sign> | <comma-symbol>) [ <space-symbol>* ]

<number-of-threads> ::= <positive-number>

<positive-number> ::= <decimal-positive-number> | <octal-number> | <hexadecimal-number>

In the syntax above, values of <MKL-domain-env-name> indicate function domains as follows:

MKL_DOMAIN_ALL

All function domains

MKL_DOMAIN_BLAS

BLAS Routines

MKL_DOMAIN_FFT

non-cluster Fourier Transform Functions

MKL_DOMAIN_LAPACK

LAPACK Routines

MKL_DOMAIN_VML

Vector Mathematics (VM)

MKL_DOMAIN_PARDISO

Intel® oneAPI Math Kernel Library PARDISO, a direct sparse solver based on Parallel Direct Sparse Solver (PARDISO*)

For example, you could set the MKL_DOMAIN_NUM_THREADSenvironment variable to any of the following string variants, in this case, defining three specific domain variables internal to Intel® oneAPI Math Kernel Library:

MKL_DOMAIN_NUM_THREADS="MKL_DOMAIN_ALL=2, MKL_DOMAIN_BLAS=1, MKL_DOMAIN_FFT=4"

MKL_DOMAIN_NUM_THREADS="MKL_DOMAIN_ALL 2 : MKL_DOMAIN_BLAS 1 : MKL_DOMAIN_FFT 4"

MKL_DOMAIN_NUM_THREADS="MKL_DOMAIN_ALL=2 : MKL_DOMAIN_BLAS=1 : MKL_DOMAIN_FFT=4"

MKL_DOMAIN_NUM_THREADS="MKL_DOMAIN_ALL=2; MKL_DOMAIN_BLAS=1; MKL_DOMAIN_FFT=4"

MKL_DOMAIN_NUM_THREADS="MKL_DOMAIN_ALL=2 MKL_DOMAIN_BLAS 1, MKL_DOMAIN_FFT 4"

MKL_DOMAIN_NUM_THREADS="MKL_DOMAIN_ALL,2: MKL_DOMAIN_BLAS 1, MKL_DOMAIN_FFT,4"

NOTE:
Prepend the appropriate set/export/setenv command for your command shell and operating system. Refer to Setting the Environment Variables for Threading Control for more details.

The global variables MKL_DOMAIN_ALL, MKL_DOMAIN_BLAS, MKL_DOMAIN_FFT, MKL_DOMAIN_VML, and MKL_DOMAIN_PARDISO, as well as the interface for the Intel® oneAPI Math Kernel Library threading control functions, can be found in themkl.h header file.

NOTE:
You can retrieve the values of the specific domain variables that you have set in your code with a call to the mkl_get_domain_max_threads(domain_name) function per the Fortran and C interface with the desired domain variable name.

This table illustrates how values of MKL_DOMAIN_NUM_THREADS are interpreted.

Value of MKL_DOMAIN_NUM_THREADS

Interpretation

MKL_DOMAIN_ALL=4

All parts of Intel® oneAPI Math Kernel Library should try four OpenMP threads. The actual number of threads may be still different because of theMKL_DYNAMIC setting or system resource issues. The setting is equivalent to MKL_NUM_THREADS = 4.

MKL_DOMAIN_ALL=1, MKL_DOMAIN_BLAS=4

All parts of Intel® oneAPI Math Kernel Library should try one OpenMP thread, except for BLAS, which is suggested to try four threads.

MKL_DOMAIN_VML=2

VM should try two OpenMP threads. The setting affects no other part of Intel® oneAPI Math Kernel Library.

Be aware that the domain-specific settings take precedence over the overall ones. For example, the "MKL_DOMAIN_BLAS=4" value of MKL_DOMAIN_NUM_THREADS suggests trying four OpenMP threads for BLAS, regardless of later setting MKL_NUM_THREADS, and a function call "mkl_domain_set_num_threads ( 4, MKL_DOMAIN_BLAS );" suggests the same, regardless of later calls to mkl_set_num_threads().
However, a function call with input "MKL_DOMAIN_ALL", such as "mkl_domain_set_num_threads (4, MKL_DOMAIN_ALL);" is equivalent to "mkl_set_num_threads(4)", and thus it will be overwritten by later calls to mkl_set_num_threads. Similarly, the environment setting of MKL_DOMAIN_NUM_THREADS with "MKL_DOMAIN_ALL=4" will be overwritten with MKL_NUM_THREADS = 2.

Whereas the MKL_DOMAIN_NUM_THREADS environment variable enables you set several variables at once, for example, "MKL_DOMAIN_BLAS=4,MKL_DOMAIN_FFT=2", the corresponding function does not take string syntax. So, to do the same with the function calls, you may need to make several calls, which in this example are as follows:

mkl_domain_set_num_threads ( 4, MKL_DOMAIN_BLAS );

mkl_domain_set_num_threads ( 2, MKL_DOMAIN_FFT );

Parent topic: Using Additional Threading Control
Level Two Title