6.11. Performing Inference on the Inflated 3D (I3D) Graph
Before you try the instructions in this section, ensure that you have completed the following tasks:
- Set up OpenVINO Model Zoo as described in Preparing a Model.
- Set up Model Optimizer as described in Preparing OpenVINO Model Zoo.
- Program the FPGA and initialize the $curarch environment variable as described in Programming the FPGA Device
Remember: A model must be converted from a framework (such as Tensorflow, Caffe, or PyTorch) into a .bin/.xml file pair before the FPGA AI Suite compiler (dla_compiler command) can ingest the model.
Preparing the i3d-rgb-tf Model
The following commands download the i3d-rgb-tf TensorFlow model and run Model Optimizer:
source ~/build-openvino-dev/openvino_env/bin/activate omz_downloader --name i3d-rgb-tf \ --output_dir $COREDLA_WORK/demo/models/ omz_converter --name i3d-rgb-tf \ --download_dir $COREDLA_WORK/demo/models/ \ --output_dir $COREDLA_WORK/demo/models/
After running these commands, you can find the model in the following directory:
demo/models/public/i3d-rgb-tf/FP32/
Inference on the i3d-rgb-tf Model and Accuracy Calculations
This section details how to run inference on i3d-rgb-tf with the sample videos and ground truth files provided. For 3D input, the batch size determines how many "clips" a video should be broken down into.
To run inference on the i3d-rgb-tf model with 3 clips per video, run the following commands:
cd $COREDLA_WORK/runtime/build_Release/dla_benchmark gnd=$COREDLA_WORK/demo/sample_videos/groundtruth_batch_size_3.txt ./dla_benchmark \ -b=3 \ -niter=4 \ -m $COREDLA_WORK/demo/models/public/i3d-rgb-tf/FP32/i3d-rgb-tf.xml \ -d=HETERO:FPGA,CPU \ -i=$COREDLA_WORK/demo/sample_videos/ \ -plugins=../../plugins.xml \ -arch_file=$curarch \ -api=async \ -nireq=4 \ -perf_est \ -groundtruth_loc=$gnd \ -folding_option=1
To run inference on the i3d-rgb-tf model with 1 clip per video, change the ground truth file (-groundtruth_loc) and the -b flag as follows:
cd $COREDLA_WORK/runtime/build_Release/dla_benchmark gnd=$COREDLA_WORK/demo/sample_videos/groundtruth_batch_size_1.txt ./dla_benchmark \ -b=1 \ -niter=4 \ -m $COREDLA_WORK/demo/models/public/i3d-rgb-tf/FP32/i3d-rgb-tf.xml \ -d=HETERO:FPGA,CPU \ -i=$COREDLA_WORK/demo/sample_videos/ \ -plugins=../../plugins.xml \ -arch_file=$curarch \ -api=async \ -nireq=4 \ -perf_est \ -groundtruth_loc=$gnd \ -folding_option=1
Important: Loading the video data with the dla_benchmark command places a high memory demand on the host. Using values for the -niter option that are less than 100 results in manageable memory requirements for most hosts. If the memory usage is too high, the operating system might abruptly end the running dla_benchmark command.
Ground truth files are included for both 3 clips per video, and 1 clip per video. To use a different number of clips per video:
- Create a new ground truth text file that contains the correct action class for each video being repeated the same amount of time as the amount of clips per video
See the groundtruth_batch_size_3.txt file for reference. The ground truths are repeated 3 times each in that file.
- Change the -groundtruth_loc option to point to the new ground truth text file that you created.
To run the dla_benchmark command on the CPU, change the ‑d=HETERO:FPGA,CPU option to ‑d=HETERO:CPU.