Intel® Math Kernel Library (Intel® MKL) PARDISO supports an out-of-core (OOC) mode that can be used to solve very large sparse systems of equations. This feature results in the storing of intermediate matrix factors in external files on the disc, thus reducing the amount of RAM required during the execution of the solver.
The parameter, iparm[59], controls whether the out-of-core (OOC) or an in-core version of the Intel MKL PARDISO direct sparse solver is used. Because the minimum amount of memory required for out-of-core execution depends on the number of OpenMP threads available, it is important that the user does not change the number of OpenMP threads used between the first call to Intel® MKL PARDISO and the factorization and solution phases.
The default value of iparm[59] is 0.
· If iparm[59] is set to 0, then the standard in-core Intel® MKL PARDISO is used.
· If iparm[59] is set to 1, then the standard in-core Intel® MKL PARDISO is used if the total memory (in MBytes) required to store the matrix factors is less than sum of the value of the two environment variables: MKL_PARDISO_OOC_MAX_CORE_SIZE (default value 2000 MB), and MKL_PARDISO_OOC_MAX_SWAP_SIZE (default value 0 MB); Intel® PARDISO OOC is used otherwise.
· If iparm[59] is set to 2, then Intel® MKL PARDISO OOC mode is used.
Note that if iparm[59] is equal to 1 or 2, and the total memory required to store internal data is more than MKL_PARDISO_OOC_MAX_CORE_SIZE, the program exits with error -9. In this case, increase the value of MKL_PARDISO_OOC_MAX_CORE_SIZE.
Default parameters for the OOC mode can be changed either by using the configuration file or by setting the environment variables via the command line.
By default, the name of the configuration file is pardiso_ooc.cfg and it should be placed in the current working directory. To change the path to the configuration file and/or its name, one can set the environment variables MKL_PARDISO_OOC_CFG_PATH for the path and MKL_PARDISO_OOC_CFG_FILE_NAME for the configuration file name. When these two environment variables are set, the path and the file name are constructed as follows:
<MKL_PARDISO_OOC_CFG_PATH >/< MKL_PARDISO_OOC_CFG_FILE_NAME> for Linux* OS , and
<MKL_PARDISO_OOC_CFG_PATH >\< MKL_PARDISO_OOC_CFG_FILE_NAME> for Windows* OS.
The list of OOC parameters that can be set by the user is given in the table below.
|
Parameter/environment variable name |
Description |
|
MKL_PARDISO_OOC_PATH |
Directory for storing data created in the OOC mode. |
|
MKL_PARDISO_OOC_FILE_NAME |
Full file name (incl. path) which will be used for the OOC files |
|
MKL_PARDISO_OOC_MAX_CORE_SIZE |
Maximum size of RAM (in megabytes) available for Intel MKL PARDISO |
|
MKL_PARDISO_OOC_MAX_SWAP_SIZE |
Maximum swap size (in megabytes) available for Intel MKL PARDISO |
|
MKL_PARDISO_OOC_KEEP_FILE |
A flag which determines whether temporary data files will be deleted or stored |
By default, the current working directory is used as a directory path for storing data. All work arrays will be stored in files named ooc_temp with different extensions. When MKL_PARDISO_OOC_FILE_NAME is not set and MKL_PARDISO_OOC_PATH is set, the names for the created files will contain <path>/mkl_pardiso or <path>\mkl_pardiso depending on the OS. Setting MKL_PARDISO_OOC_FILE_NAME=<filename> will override the path which could have been set in MKL_PARDISO_OOC_PATH. In this case <filename> will be used for naming the OOC files.
Also, by default, MKL_PARDISO_OOC_MAX_CORE_SIZE is 2000 (MB) and MKL_PARDISO_OOC_MAX_SWAP_SIZE is 0.
Notice: do not set the sum of MKL_PARDISO_OOC_MAX_CORE_SIZE and MKL_PARDISO_OOC_MAX_SWAP_SIZE greater than the size of the RAM plus the size of the swap. Be sure to allow enough free memory for the operating system and any other processes which need to be running.
By default, all temporary data files will be deleted. For keeping them it is required to set MKL_PARDISO_OOC_KEEP_FILE to 0.
An example of the configuration file with OOC parameters:
MKL_PARDISO_OOC_PATH = <path>
MKL_PARDISO_OOC_MAX_CORE_SIZE = N
MKL_PARDISO_OOC_MAX_SWAP_SIZE = K
MKL_PARDISO_OOC_KEEP_FILE = 0 (or 1)
CAUTION: The maximum length of the path lines in the configuration files is 1000 characters.
If the configuration file cannot be found, the solver will read the values set for the environment variables.
An example of setting environment variables:
For Linux* OS and OS X*:
export MKL_PARDISO_OOC_PATH = <path>
export MKL_PARDISO_OOC_MAX_CORE_SIZE = N
export MKL_PARDISO_OOC_MAX_SWAP_SIZE = K
export MKL_PARDISO_OOC_KEEP_FILE = 0 (or 1)
For Windows* OS:
set MKL_PARDISO_OOC_PATH = <path>
set MKL_PARDISO_OOC_MAX_CORE_SIZE = N
set MKL_PARDISO_OOC_MAX_SWAP_SIZE = K
set MKL_PARDISO_OOC_KEEP_FILE = 0 (or 1)
where <path> should follow the OS naming convention.
The internal two-level factorization algorithm will not improve performance when Intel MKL PARDISO is used in OOC mode.
The minimum amount of OOC memory used for numerical factorization and solution can be modified. The total memory consumption of OOC mode can be estimated by max(iparm[15], iparm[16]+iparm[63]), Alternatively, a user can also extract the minimum amount of memory required by PARDISO OOC in kilobytes as the return value of iparm[63] after computing reordering phase.
Please refer to the Intel® MKL Developer Reference for more details about Intel® MKL PARDISO OOC mode and to the KB article Tips For Using Intel(R) MKL PARDISO.