Introducing the Intel® Neural Compute Stick 2

Ashwin Vijaya Kumar

More hardware acceleration for your neural networks

The Intel® Neural Compute Stick 2 (Intel® NCS 2) is Intel’s newest deep learning inference development kit. Packed in an affordable USB-stick form factor, the Intel® NCS 2 is powered by our latest VPU (vision processing unit) – the Intel® Movidius™ Myriad X, which includes an on-chip neural network accelerator called the Neural Compute Engine. With 16 SHAVE cores and a dedicated hardware neural network accelerator, the NCS 2 offers up to 8x performance improvement⁺ over the previous generation.

Software tools to accelerate deep learning inference

The Intel® Distribution of OpenVINO™ toolkit is the default software development kit¹ to optimize performance, integrate deep learning inference and run deep neural networks (DNN) on Intel® Movidius™ Vision Processing Units (VPU). (For the previous generation, developers used the Intel® Movidius NCS SDK). This toolkit supports a broad set of neural networks and streamlines deployment across not only NCS 2 hardware, but the full range of Intel vision accelerator solutions². At the time of writing this article, this toolkit supports more than 20 pre-trained models³ covering image classification, object detection and image segmentation.

Develop on one platform, deploy across multiple

That's the mantra and simple elegance of the Intel Distribution of OpenVINO toolkit. Thanks to an intermediate representation (IR) format, you can develop and test a neural network on one type of processor such as a CPU, and deploy the same model on a range of processing units such as Intel® processors (CPU, GPU/Intel® Processor Graphics, VPU, FPGA) or even deploy heterogeneously (splitting the model) across two processors⁴. The IR concept allows you to run models built using multiple frameworks⁵ such as TensorFlow™, Caffe*, and MXNet*, and other exchange formats like ONNX*. This flexibility of supporting multiple frameworks, exchange formats and hardware accelerators is made possible due to the toolkit’s modular architecture. Below is a simplified graphical representation of the toolkit’s software components.

OpenVINO Composition

Intel® Distribution of OpenVINO™ Toolkit

Streamlined and easy development workflow

The toolkit has a simple development workflow, and it only takes three steps to develop and deploy a neural network on any of the supported processors and accelerators.

Train a model on your preferred training hardware using one of the supported frameworks⁵.
- You can choose to use one of the many pre-trained models³ shipped with the toolkit.
Convert the trained model into a IR file using the toolkit’s model optimizer.
Offload the IR model onto one of the supported hardware accelerators⁶ to perform inference.

OpenVINO workflow

Toolkit Workflow

This article walks you through the process of building your first artificial intelligence (AI) app using pre-trained neural networks, Intel Distribution of OpenVINO toolkit, and the Intel NCS 2.

Practical Learning!

You will build...

A set of AI apps that can perform image classification⁷, object detection⁸ and image segmentation⁹ on Intel NCS 2.

You will learn...

How to install and configure Intel Distribution of OpenVINO toolkit to support Intel NCS 2
How to deploy a pre-trained neural network on Intel NCS 2

You will need...

An Intel® Neural Compute Stick 2 - Where to buy
An x86_64 laptop/desktop running Ubuntu 16.04 ("Development machine")

If not already done so, follow the instructions on Intel NCS 2 getting started guide to install the toolkit and Intel NCS 2 plugins on your development machine.

Let's build!

Running the toolkit's pre-trained models on Intel NCS 2

If you followed the Intel NCS 2 getting started guide, you have already run some of the pre-trained models on the Intel NCS 2. This confirms that your development machine is fully set up to convert pre-trained models to IR files, and deploy these IR files onto Intel NCS 2 using the toolkit’s inference engine API. You probably ran one example each for image classification, object detection and image segmentation.

Running publicly available pre-trained models on Intel NCS 2

The really cool thing about the deep learning community is that they have published several pre-trained models for free. Let's do our part in preventing global warming by not duplicating their efforts in re-training the same network on the same dataset that would consume power for a week or two. The below steps walk you through the toolkit's development workflow of converting publicly available pre-trained neural net models to IR files and then running them on Intel NCS 2.

Run the following commands in a terminal window

On most Linux machines, hitting `ctrl+alt+t` will open a terminal window.

Step 1: Download publicly available models that are known to work with the toolkit

cd ~/intel/computer_vision_sdk/deployment_tools/model_downloader

# List public models that are known to work with OpenVINO
python3 downloader.py --print_all

# Download a specific model, say GoogLeNet V2
python3 downloader.py --name googlenet-v2

You can run downloader.py without the --name option to download all models, but it'll take quite a while.

If the script ran fine, you should see `googlenet-v2.caffemodel` and `googlenet-v2.prototxt` in `model_downloader/classification/googlenet/v2/caffe` folder.

Step 2: Convert the downloaded pre-trained model into IR files

cd ~/intel/computer_vision_sdk/deployment_tools/model_downloader/classification/googlenet/v2/caffe

# Ensure that the OpenVINO environment is initialized
source ~/intel/computer_vision_sdk/bin/setupvars.sh

# Use model optimizer to convert googlenet.caffemodel to IR
mo.py --data_type FP16 --input_model googlenet-v2.caffemodel --input_proto googlenet-v2.prototxt

If the script ran fine, you should see `googlenet-v2.bin`, `googlenet-v2.mapping` and `googlenet-v2.xml` in `model_downloader/classification/googlenet/v2/caffe` folder.

Step 3: Deploy the converted IR model onto Intel NCS 2 using the toolkit's IE API

cd ~/intel/computer_vision_sdk/deployment_tools/inference_engine/samples/python_samples

# Download a test image from the internet
wget -N https://upload.wikimedia.org/wikipedia/commons/b/b6/Felis_catus-cat_on_snow.jpg

# Ensure that the OpenVINO environment is initialized
source ~/intel/computer_vision_sdk/bin/setupvars.sh

# Run an inference on this image using a built-in sample code
python3 classification_sample.py -m ~/intel/computer_vision_sdk/deployment_tools/model_downloader/classification/googlenet/v2/caffe/googlenet-v2.xml -i Felis_catus-cat_on_snow.jpg -d MYRIAD

If everything ran fine, you should see the below message in the terminal window. Class #173 corresponds to the 'tabby cat' class/category. Try downloading other images of tabby cat and rerunning the example.

Image Felis_catus-cat_on_snow.jpg

0.3881836 label #173
0.3193359 label #54
0.2410889 label #7
0.0361328 label #200
0.0037460 label #84
0.0025158 label #66
0.0021381 label #10
0.0016766 label #473
0.0013685 label #198
0.0007257 label #152

Congratulations! You have successfully installed and configured Intel Distribution of OpenVINO Toolkit to develop smart apps for Intel® Neural Compute Stick 2.

Further Experiments

Okay, you just ran what I call 'the blinky code' (or 'Hello World!') example, now what? How about a 'my-first-openvino-app'? Try writing your own application using the Inference Engine API.
At the time of writing this blog, the toolkit did not ship with a labels file which could be used to display a sensible inference result as against a class ID like #173. Try using the `--labels` flag with `classification_sample.py` to print out actual names of the inference results.
- Reference: See "BEST REPLY" in this community forum post.

Select Your Language

Using Intel.com Search

Quick Links

Recent Searches

Advanced Search

Only search in

Introducing the Intel® Neural Compute Stick 2 - Intel's Next Generation AI Inference Development Kit