Heterogeneous Plugin with the Intel® Neural Compute Stick 2

Documentation

Install & Setup

000055368

09/07/2022

The Heterogeneous Plugin (HETERO) allows computing inference on one network using several devices. You may want to execute networks in heterogeneous mode in order to:

  • Utilize the power of accelerators, including calculating the heaviest parts of the network on accelerators, and execute unsupported layers on fallback devices like the CPU.
  • Utilize all available hardware more efficiently during one inference.

To use the Heterogeneous Plugin and run inference on your Intel® Neural Compute Stick 2 using OpenVINO’s™ Benchmark application, follow the steps below:

  1. Build the C++ samples by running buildsamples.sh (for Linux) or build_samples_msvc.bat (for Windows) found in inference_engine/samples directory.
  2. Use the benchmark_app to test and configure the Hetero Plugin with the devices that you would like to use. For example, running -d HETERO:GPU,CPU will give priority to GPU, while CPU will be a fallback device.

Find more information about the Heterogeneous Plugin and the OpenVINO Toolkit.