OpenVINO™ inference time increases when running multiple processes

The inferencing time doubles when running two processes to infer the same model.

• Using OpenVINO™ to infer a model.
• Inference time is about 300ms when running a single process.
• When using two processes, the inference time for both processes becomes 600ms.

1. Add the following command in the main.cpp script.

   ie.SetConfig({ { CONFIG_KEY(CPU_BIND_THREAD), "NO" } }, "CPU")

2. Build the demo.

• The default value of the KEY_CPU_BIND_THREAD config is YES.
• On Linux*, this causes the two processes to bind to the same CPU core and doubles inference time.
• Setting the KEY_CPU_BIND_THREAD config to NO unbinds inferencing threads from CPU cores on both Windows* and Linux*.

Refer to Supported Configuration Parameters for more information on the configuration parameters for the CPU plugin.