Constant Tensor Cache

Intel® oneAPI Deep Neural Network Developer Guide and Reference

Download PDF

ID 768875

Date 6/20/2024

Version

Public

A newer version of this document is available. Customers should click here to go to the newest version.

Visible to Intel only — GUID: GUID-07E96FC3-A157-41F5-93CE-32CCDAD8FC3D

View Details

Constant Tensor Cache

Overview

A set of functions that provide constant tensor cache control. More…

// global functions

void dnnl::graph::set_constant_tensor_cache(int flag);
int dnnl::graph::get_constant_tensor_cache();

void dnnl::graph::set_constant_tensor_cache_capacity(
    engine::kind kind,
    size_t size
    );

size_t dnnl::graph::get_constant_tensor_cache_capacity(engine::kind kind);
dnnl_status_t DNNL_API dnnl_graph_set_constant_tensor_cache(int flag);
dnnl_status_t DNNL_API dnnl_graph_get_constant_tensor_cache(int* flag);

dnnl_status_t DNNL_API dnnl_graph_set_constant_tensor_cache_capacity(
    dnnl_engine_kind_t eng_kind,
    size_t size
    );

dnnl_status_t DNNL_API dnnl_graph_get_constant_tensor_cache_capacity(
    dnnl_engine_kind_t eng_kind,
    size_t* size
    );

Detailed Documentation

A set of functions that provide constant tensor cache control.

Global Functions

void dnnl::graph::set_constant_tensor_cache(int flag)

Control the enabling or disabling of constant tensor cache.

This API must be called once before compilation stage. By default, constant tensor cache is disabled in the library. .. note:

This API is deprecated and will be removed in future release, please use the set_constant_tensor_cache_capacity API to disable constant tensor cache by setting it's capacity to zero.

Parameters:

flag	Set to positive value to enable the cache and set to 0 to disable the cache. Negative values are invalid.

int dnnl::graph::get_constant_tensor_cache()

Return the enabling status of constant tensor cache.

NOTE:

This API is deprecated and will be removed in future release, please use the get_constant_tensor_cache_capacity API to check the enabling status by checking it’s capacity.

void dnnl::graph::set_constant_tensor_cache_capacity(
    engine::kind kind,
    size_t size
    )

Control the capacity for the constant tensor cache that used for specific engine kind.

This API is thread safe and can be called multiple times at runtime. The capacity is set to zero by default which means the cache is disabled. When calling this API, the corresponding cache will be flushed. Setting capacity to 0 means to clear all cached tensors and disable cache. Once the capacity limit is reached, no new tensors will be cached. If there are multiple devices for an engine kind, the capacity set here is for each device.

Parameters:

kind	The engine kind that the constant tensor cache used for.
size	The constant tensor cache capacity size to set.

size_t dnnl::graph::get_constant_tensor_cache_capacity(engine::kind kind)

Return the current capacity of constant tensor cache.

Parameters:

kind	The engine kind that the constant tensor cache used for.

dnnl_status_t DNNL_API dnnl_graph_set_constant_tensor_cache(int flag)

Control the enabling or disabling of constant tensor cache.

This API must be called once before compilation stage. By default, constant tensor cache is disabled in the library.

NOTE:

This API is deprecated and will be removed in future release, please use the dnnl_graph_set_constant_tensor_cache_capacity API to disable constant tensor cache by setting it’s capacity to zero.

Parameters:

flag	Set to positive value to enable the cache and set to 0 to disable the cache. Negative values are invalid.

Returns:

dnnl_invalid_arguments if the flag value is invalid, and dnnl_success on success.

dnnl_status_t DNNL_API dnnl_graph_get_constant_tensor_cache(int* flag)

Return the enabling or disabling status of constant tensor cache.

NOTE:

This API is deprecated and will be removed in future release, please use the dnnl_graph_get_constant_tensor_cache_capacity API to check the enabling status by checking it’s capacity.

Parameters:

flag	The constant tensor cache enabling status to query.

Returns:

dnnl_invalid_arguments if the flag value is nullptr, and dnnl_success on success.

dnnl_status_t DNNL_API dnnl_graph_set_constant_tensor_cache_capacity(
    dnnl_engine_kind_t eng_kind,
    size_t size
    )

Control the capacity for the constant tensor cache that used for specific engine kind.

Parameters:

eng_kind	The engine kind that the constant tensor cache used for.
size	The constant tensor cache capacity size to set.

Returns:

dnnl_invalid_arguments if the eng_kind value is invalid, and dnnl_success on success.

dnnl_status_t DNNL_API dnnl_graph_get_constant_tensor_cache_capacity(
    dnnl_engine_kind_t eng_kind,
    size_t* size
    )

Return the current capacity of constant tensor cache.

Parameters:

eng_kind	The engine kind that the constant tensor cache used for.
size	The constant tensor cache capacity size to query.

Returns:

dnnl_invalid_arguments if the eng_kind value is nullptr or the size is nullptr, and dnnl_success on success.

Select Your Language

Using Intel.com Search

Quick Links

Recent Searches

Advanced Search

Only search in

Intel® oneAPI Deep Neural Network Developer Guide and Reference

Constant Tensor Cache

Overview

Detailed Documentation