Developer Guide

Developer Guide for Intel® oneAPI Math Kernel Library Linux*

Download PDF
ID 766690
Date 3/22/2024
Public

A newer version of this document is available. Customers should click here to go to the newest version.

Document Table of Contents
Document Table of Contents x
Developer Guide for Intel® oneAPI Math Kernel Library for Linux*
Developer Guide for Intel® oneAPI Math Kernel Library for Linux* x
Getting Help and Support What's New Notational Conventions Related Information Getting Started Structure of the Intel® oneAPI Math Kernel Library Linking Your Application with the Intel® oneAPI Math Kernel Library Managing Performance and Memory Language-specific Usage Options Obtaining Numerically Reproducible Results Coding Tips Managing Output Working with the Intel® oneAPI Math Kernel Library Cluster Software Managing Behavior of the Intel® oneAPI Math Kernel Library with Environment Variables Configuring Your Integrated Development Environment to Link with Intel® oneAPI Math Kernel Library Intel® oneAPI Math Kernel Library Benchmarks Appendix A: Intel® oneAPI Math Kernel Library Language Interfaces Support Appendix B: Support for Third-Party Interfaces Appendix C: Directory Structure in Detail Notices and Disclaimers
Getting Started x
Shared Library Versioning CMake Config for oneMKL Checking Your Installation Setting Environment Variables Compiler Support Using Code Examples What You Need to Know Before You Begin Using the Intel® oneAPI Math Kernel Library
Setting Environment Variables x
Scripts to Set Environment Variables Modulefiles to Set Environment Variables Automating the Process of Setting Environment Variables Using the CMake Config File
Structure of the Intel® oneAPI Math Kernel Library x
Architecture Support High-Level Directory Structure Layered Model Concept
Linking Your Application with the Intel® oneAPI Math Kernel Library x
Linking Quick Start Linking Examples Linking in Detail Building Custom Shared Objects
Linking Quick Start x
Using the -qmkl Compiler Option Using the -qmkl-ilp64 Compiler Option Using the Single Dynamic Library Selecting Libraries to Link with Using the Link-line Advisor Using the Command-Line Link Tool
Linking Examples x
Linking on IA-32 Architecture Systems Linking on Intel(R) 64 Architecture Systems
Linking in Detail x
Listing Libraries on a Link Line Dynamically Selecting the Interface and Threading Layer Linking with Interface Libraries Linking with Threading Libraries Linking with Computational Libraries Linking with Compiler Run-time Libraries Linking with System Libraries
Linking with Interface Libraries x
Using the ILP64 Interface vs. LP64 Interface Linking with Fortran 95 Interface Libraries
Building Custom Shared Objects x
Using the Custom Shared Object Builder Composing a List of Functions Specifying Function Names Distributing Your Custom Shared Object
Managing Performance and Memory x
Improving Performance with Threading Improving Performance for Small Size Problems Other Tips and Techniques to Improve Performance Using Memory Functions
Improving Performance with Threading x
OpenMP* Threaded Functions and Problems Functions Threaded with Intel® Threading Building Blocks Avoiding Conflicts in the Execution Environment Techniques to Set the Number of Threads Setting the Number of Threads Using an OpenMP* Environment Variable Changing the Number of OpenMP* Threads at Run Time Using Additional Threading Control Calling oneMKL Functions from Multi-threaded Applications Using Intel® Hyper-Threading Technology Managing Multi-core Performance Managing Performance with Heterogeneous Cores
Using Additional Threading Control x
oneMKL-specific Environment Variables for OpenMP Threading Control MKL_DYNAMIC MKL_DOMAIN_NUM_THREADS MKL_NUM_STRIPES Setting the Environment Variables for Threading Control
Improving Performance for Small Size Problems x
Using MKL_DIRECT_CALL in C Applications Using MKL_DIRECT_CALL in Fortran Applications Limitations of the Direct Call
Other Tips and Techniques to Improve Performance x
Coding Techniques Improving oneMKL Performance on Specific Processors Operating on Denormals
Using Memory Functions x
Avoiding Memory Leaks in oneMKL Redefining Memory Functions
Language-specific Usage Options x
Using Language-Specific Interfaces with Intel® oneAPI Math Kernel Library Mixed-language Programming with the Intel Math Kernel Library
Using Language-Specific Interfaces with Intel® oneAPI Math Kernel Library x
Interface Libraries and Modules Fortran 95 Interfaces to LAPACK and BLAS Compiler-dependent Functions and Fortran 90 Modules
Mixed-language Programming with the Intel Math Kernel Library x
Calling LAPACK, BLAS, and CBLAS Routines from C/C++ Language Environments Using Complex Types in C/C++ Calling BLAS Functions That Return the Complex Values in C/C++ Code
Obtaining Numerically Reproducible Results x
Getting Started with Conditional Numerical Reproducibility Specifying Code Branches Reproducibility Conditions Setting the Environment Variable for Conditional Numerical Reproducibility Code Examples
Coding Tips x
Example of Data Alignment Using Predefined Preprocessor Symbols for Intel® MKL Version-Dependent Compilation
Managing Output x
Using oneMKL Verbose Mode
Using oneMKL Verbose Mode x
Version Information Line Call Description Line Call Description Line for CPU Call Description Line for GPU
Working with the Intel® oneAPI Math Kernel Library Cluster Software x
Linking with oneMKL Cluster Software Setting the Number of OpenMP* Threads Using Shared Libraries Setting Environment Variables on a Cluster Interaction with the Message-passing Interface Using a Custom Message-Passing Interface Examples of Linking for Clusters
Examples of Linking for Clusters x
Examples for Linking a C Application Examples for Linking a Fortran Application
Managing Behavior of the Intel® oneAPI Math Kernel Library with Environment Variables x
Managing Behavior of Function Domains with Environment Variables Instruction Set–Specific Dispatching on Intel® Architectures
Managing Behavior of Function Domains with Environment Variables x
Setting the Default Mode of Vector Math with an Environment Variable Managing Performance of the Cluster Fourier Transform Functions Managing Invalid Input Checking in LAPACKE Functions
Configuring Your Integrated Development Environment to Link with Intel® oneAPI Math Kernel Library x
Configuring the Eclipse* IDE CDT to Link with oneMKL
Intel® oneAPI Math Kernel Library Benchmarks x
Intel Optimized LINPACK Benchmark for Linux* Intel® Distribution for LINPACK* Benchmark and Intel® Optimized HPL-AI* Benchmark Intel® Optimized High Performance Conjugate Gradient Benchmark
Intel Optimized LINPACK Benchmark for Linux* x
Contents of the Intel® Optimized LINPACK Benchmark Running the Software Known Limitations of the Intel® Optimized LINPACK Benchmark
Intel® Distribution for LINPACK* Benchmark and Intel® Optimized HPL-AI* Benchmark x
Overview of the Intel® Distribution for LINPACK* Benchmark Overview of the Intel® Optimized HPL-AI* Benchmark Contents of the Intel® Distribution for LINPACK* Benchmark and the Intel® Optimized HPL-AI* Benchmark Building the Intel® Distribution for LINPACK* Benchmark and the Intel® Optimized HPL-AI* Benchmark for a Customized MPI Implementation Building the Netlib HPL from Source Code Configuring Parameters Ease-of-use Command-line Parameters Running the Intel® Distribution for LINPACK* Benchmark and the Intel® Optimized HPL-AI* Benchmark Heterogeneous Support in the Intel® Distribution for LINPACK* Benchmark Environment Variables Improving Performance of Your Cluster
Intel® Optimized High Performance Conjugate Gradient Benchmark x
Overview of the Intel Optimized HPCG Versions of the Intel® CPU Optimized HPCG Versions of the Intel® GPU Optimized HPCG Getting Started with Intel® CPU Optimized HPCG Getting Started with Intel® GPU Optimized HPCG Choosing the Best Configuration and Problem Sizes for CPUs Choosing the Best HPCG Configuration for GPUs
Appendix A: Intel® oneAPI Math Kernel Library Language Interfaces Support x
Language Interfaces Support, by Function Domain Include Files
Appendix B: Support for Third-Party Interfaces x
FFTW Interface Support
Appendix C: Directory Structure in Detail x
Detailed Structure of the IA-32 Architecture Directories Detailed Structure of the Intel® 64 Architecture Directories
Detailed Structure of the IA-32 Architecture Directories x
Static Libraries in the lib32 Directory Dynamic Libraries in the lib32 Directory
Detailed Structure of the Intel® 64 Architecture Directories x
Static Libraries in the lib Directory Dynamic Libraries in the lib Directory
Developer Guide for Intel® oneAPI Math Kernel Library for Linux*

Call Description Line

Call Description Line for CPU

In Intel® oneAPI Math Kernel Library (oneMKL) Verbose mode, each verbose-enabled function called from your application prints a call description line. The line begins with the MKL_VERBOSE character string and uses spaces as delimiters. The format of the rest of the line is subject to change in a future release.

The following table lists information contained in a call description line for Verbose with CPU applications and provides available links for more information:

Information

Description

Related Links

The name of the function.

Although the name printed may differ from the name used in the source code of the application (for example, the cblas_ prefix of CBLAS functions is not printed), you can easily recognize the function by the printed name.

 

Values of the arguments.

  • The values are listed in the order of the formal argument list. The list directly follows the function name, it is parenthesized and comma-separated.
  • Arrays are printed as addresses (to see the alignment of the data).
  • Integer scalar parameters passed by reference are printed by value. Zero values are printed for NULL references.
  • Character values are printed without quotes.
  • For all parameters passed by reference, the values printed are the values returned by the function. For example, the printed value of the info parameter of a LAPACK function is its value after the function execution.
  • For verbose-enabled functions in the ScaLAPACK domain, in addition to the standard input parameters, information about blocking factors, MPI rank, and process grid is also printed.
 

Time taken by the function.

  • The time is printed in convenient units (seconds, milliseconds, and so on), which are explicitly indicated.

  • The time may fluctuate from run to run.

  • The time printed may occasionally be larger than the time actually taken by the function call, especially for small problem sizes and multi-socket machines.To reduce this effect, bind threads that call Intel® oneAPI Math Kernel Library (oneMKL) to CPU cores by setting an affinity mask.

Managing Multi-core Performance for options to set an affinity mask.

Value of the MKL_CBWR environment variable.

The value printed is prefixed with CNR:

Getting Started with Conditional Numerical Reproducibility

Value of the MKL_DYNAMIC environment variable.

The value printed is prefixed with Dyn:

MKL_DYNAMIC

Status of the Intel® oneAPI Math Kernel Library (oneMKL)memory manager.

The value printed is prefixed with FastMM:

Avoiding Memory Leaks in oneMKLfor a description of the Intel® oneAPI Math Kernel Library (oneMKL)memory manager

OpenMP* thread number of the calling thread.

The value printed is prefixed with TID:

  

Values of Intel® oneAPI Math Kernel Library (oneMKL) environment variables defining the general and domain-specific numbers of threads, separated by a comma.

The first value printed is prefixed with NThr:

oneMKL-specific Environment Variables for Threading Control

The following is an example of a call description line (with OpenMP threading):

MKL_VERBOSE DGEMM(n,n,1000,1000,240,0x7ffff708bb30,0x7ff2aea4c000,1000,0x7ff28e92b000,240,0x7ffff708bb38,0x7ff28e08d000,1000) 1.66ms CNR:OFF Dyn:1 FastMM:1 TID:0 NThr:16

The following is an example of a call description line (with TBB threading):

MKL_VERBOSE DGEMM(n,n,1000,1000,240,0x7ffff708bb30,0x7ff2aea4c000,1000,0x7ff28e92b000,240,0x7ffff708bb38,0x7ff28e08d000,1000) 1.66ms CNR:OFF Dyn:1 FastMM:1

NOTE:
For more information about selected threading, refer to Version Information Line.

The following information is not printed because of limitations of Intel® oneAPI Math Kernel Library (oneMKL) Verbose mode:

  • Input values of parameters passed by reference if the values were changed by the function.

    For example, if a LAPACK function is called with a workspace query, that is, the value of the lwork parameter equals -1 on input, the call description line prints the result of the query and not -1.

  • Return values of functions.

    For example, the value returned by the function ilaenv is not printed.

  • Floating-point scalars passed by reference.

Call Description Line for GPU

In Intel® oneAPI Math Kernel Library (oneMKL) Verbose mode, each verbose-enabled function called from your application prints a call description line. The line begins with the MKL_VERBOSE character string and uses spaces as delimiters. The format of the rest of the line may change in a future release.

The following table lists information contained in a call description line for verbose with GPU applications.

See Also:
GPU Information Line
Information Description
The name of the function Although the name printed may differ from the name used in the source code of the application, you can easily recognize the function by the printed name.
The values of the arguments
  • The values are listed in the order of the formal argument list. The list directly follows the function name, and it is parenthesized and comma-separated.
  • Arrays are printed as addresses (to show the alignment of the data).
  • Integer scalar parameters passed by reference are printed by value. Zero values are printed for NULL references.
  • Character values are printed without quotation marks.
  • For all parameters passed by reference, the values printed are the values returned by the function.
Time taken by the function
  • If verbose is enabled with timing for GPU applications, kernel executions will become synchronous (previous kernel will block later kernels) and the measured time may include potential data transfers and/or data copies in host and devices.
  • If Verbose is enabled without timing for GPU applications, time will be printed out as 0.
  • The time is printed in convenient units (seconds, milliseconds, and so on), which are explicitly indicated.
  • The time may fluctuate from run to run.
  • The time printed may occasionally be larger than the time actually taken by the function call, especially for small problem sizes.
Device index

The index of the GPU device on which the kernel is being executed will be printed after the character string "GPU" (e.g. GPU0, GPU1, GPU2, etc). Use the index and refer to the GPU information lines for more information about the specific device.

If the kernel is executed on the host CPU, this field will be empty.

The following is an example of a call description line: 

MKL_VERBOSE FFT(dcfi64) 224.30us GPU0
Some Limitations:

For GPU applications, the call description lines may be printed out-of-order (the order of the call description lines printed in the verbose output may not be the order in which the kernels are submitted in the functions) for the following two cases:

  • Verbose is enabled without timing and the kernel executions stay asynchronous.
  • The kernel is not executed on one of the GPU devices, but on the host CPU (the device index will not be printed in this case).
Parent topic: Using oneMKL Verbose Mode
Level Two Title