Summary
Quick step to bind multiple inferencing processes to different CPUs
Description
The inferencing time doubles when running two processes to infer the same model.
- Using OpenVINO™ to infer a model.
- Inference time is about 300ms when running a single process.
- When using two processes, the inference time for both processes becomes 600ms.
Resolution
- Add the following command in the main.cpp script.
ie.SetConfig({ { CONFIG_KEY(CPU_BIND_THREAD), "NO" } }, "CPU")
- Build the demo.
Additional information
- The default value of the KEY_CPU_BIND_THREAD config is YES.
- On Linux*, this causes the two processes to bind to the same CPU core and doubles inference time.
- Setting the KEY_CPU_BIND_THREAD config to NO unbinds inferencing threads from CPU cores on both Windows* and Linux*.
Refer to Supported Configuration Parameters for more information on the configuration parameters for the CPU plugin.