Intel® oneAPI Deep Neural Network Developer Guide and Reference
A newer version of this document is available. Customers should click here to go to the newest version.
Visible to Intel only — GUID: GUID-07E96FC3-A157-41F5-93CE-32CCDAD8FC3D
Visible to Intel only — GUID: GUID-07E96FC3-A157-41F5-93CE-32CCDAD8FC3D
Constant Tensor Cache
Overview
A set of functions that provide constant tensor cache control. More…
// global functions void dnnl::graph::set_constant_tensor_cache(int flag); int dnnl::graph::get_constant_tensor_cache(); void dnnl::graph::set_constant_tensor_cache_capacity( engine::kind kind, size_t size ); size_t dnnl::graph::get_constant_tensor_cache_capacity(engine::kind kind); dnnl_status_t DNNL_API dnnl_graph_set_constant_tensor_cache(int flag); dnnl_status_t DNNL_API dnnl_graph_get_constant_tensor_cache(int* flag); dnnl_status_t DNNL_API dnnl_graph_set_constant_tensor_cache_capacity( dnnl_engine_kind_t eng_kind, size_t size ); dnnl_status_t DNNL_API dnnl_graph_get_constant_tensor_cache_capacity( dnnl_engine_kind_t eng_kind, size_t* size );
Detailed Documentation
A set of functions that provide constant tensor cache control.
Global Functions
void dnnl::graph::set_constant_tensor_cache(int flag)
Control the enabling or disabling of constant tensor cache.
This API must be called once before compilation stage. By default, constant tensor cache is disabled in the library. .. note:
This API is deprecated and will be removed in future release, please use the set_constant_tensor_cache_capacity API to disable constant tensor cache by setting it's capacity to zero.
Parameters:
flag |
Set to positive value to enable the cache and set to 0 to disable the cache. Negative values are invalid. |
int dnnl::graph::get_constant_tensor_cache()
Return the enabling status of constant tensor cache.
void dnnl::graph::set_constant_tensor_cache_capacity( engine::kind kind, size_t size )
Control the capacity for the constant tensor cache that used for specific engine kind.
This API is thread safe and can be called multiple times at runtime. The capacity is set to zero by default which means the cache is disabled. When calling this API, the corresponding cache will be flushed. Setting capacity to 0 means to clear all cached tensors and disable cache. Once the capacity limit is reached, no new tensors will be cached. If there are multiple devices for an engine kind, the capacity set here is for each device.
Parameters:
kind |
The engine kind that the constant tensor cache used for. |
size |
The constant tensor cache capacity size to set. |
size_t dnnl::graph::get_constant_tensor_cache_capacity(engine::kind kind)
Return the current capacity of constant tensor cache.
Parameters:
kind |
The engine kind that the constant tensor cache used for. |
dnnl_status_t DNNL_API dnnl_graph_set_constant_tensor_cache(int flag)
Control the enabling or disabling of constant tensor cache.
This API must be called once before compilation stage. By default, constant tensor cache is disabled in the library.
Parameters:
flag |
Set to positive value to enable the cache and set to 0 to disable the cache. Negative values are invalid. |
Returns:
dnnl_invalid_arguments if the flag value is invalid, and dnnl_success on success.
dnnl_status_t DNNL_API dnnl_graph_get_constant_tensor_cache(int* flag)
Return the enabling or disabling status of constant tensor cache.
Parameters:
flag |
The constant tensor cache enabling status to query. |
Returns:
dnnl_invalid_arguments if the flag value is nullptr, and dnnl_success on success.
dnnl_status_t DNNL_API dnnl_graph_set_constant_tensor_cache_capacity( dnnl_engine_kind_t eng_kind, size_t size )
Control the capacity for the constant tensor cache that used for specific engine kind.
This API is thread safe and can be called multiple times at runtime. The capacity is set to zero by default which means the cache is disabled. When calling this API, the corresponding cache will be flushed. Setting capacity to 0 means to clear all cached tensors and disable cache. Once the capacity limit is reached, no new tensors will be cached. If there are multiple devices for an engine kind, the capacity set here is for each device.
Parameters:
eng_kind |
The engine kind that the constant tensor cache used for. |
size |
The constant tensor cache capacity size to set. |
Returns:
dnnl_invalid_arguments if the eng_kind value is invalid, and dnnl_success on success.
dnnl_status_t DNNL_API dnnl_graph_get_constant_tensor_cache_capacity( dnnl_engine_kind_t eng_kind, size_t* size )
Return the current capacity of constant tensor cache.
Parameters:
eng_kind |
The engine kind that the constant tensor cache used for. |
size |
The constant tensor cache capacity size to query. |
Returns:
dnnl_invalid_arguments if the eng_kind value is nullptr or the size is nullptr, and dnnl_success on success.