FPGA AI Suite: Getting Started Guide

Download PDF
ID 768970
Date 9/06/2024
Public
Document Table of Contents
Document Table of Contents x
1. FPGA AI Suite Getting Started Guide 2. FPGA AI Suite Components 3. FPGA AI Suite Installation Overview 4. Installing the FPGA AI Suite Compiler and IP Generation Tools 5. Installing the FPGA AI Suite PCIe-Based Design Example Prerequisites 6. FPGA AI Suite Quick Start Tutorial 7. Running the Hostless DDR-Free Design Example A. FPGA AI Suite Getting Started Guide Archives B. FPGA AI Suite Getting Started Guide Document Revision History
3. FPGA AI Suite Installation Overview x
3.1. FPGA AI Suite Directory Structure
4. Installing the FPGA AI Suite Compiler and IP Generation Tools x
4.1. Supported FPGA Families 4.2. Operating System Prerequisites 4.3. Installing the FPGA AI Suite with System Package Management Tools 4.4. Installing OpenVINO™ Toolkit 4.5. Installing Quartus® Prime Pro Edition Software 4.6. Setting Required Environment Variables 4.7. Installing Intel® Threading Building Blocks (TBB) 4.8. Finalizing Your FPGA AI Suite Installation
4.3. Installing the FPGA AI Suite with System Package Management Tools x
4.3.1. Installing the FPGA AI Suite Into an Alternative Location
5. Installing the FPGA AI Suite PCIe-Based Design Example Prerequisites x
5.1. PCIe-based Design Example Hardware Prerequisites 5.2. PCIe-based Design Example Operating System Prerequisites 5.3. PCIe-based Design Example Software Prerequisites
5.3. PCIe-based Design Example Software Prerequisites x
5.3.1. Additional Software Prerequisites for the PCIe-based Design Example for Agilex™ 7 Devices
6. FPGA AI Suite Quick Start Tutorial x
6.1. Creating a Working Directory 6.2. Preparing OpenVINO™ Model Zoo 6.3. Preparing a Model 6.4. Running the Graph Compiler 6.5. Preparing an Image Set 6.6. Programming the FPGA Device 6.7. Performing Inference on the PCIe-Based Example Design 6.8. Building an FPGA Bitstream for the PCIe Example Design 6.9. Building the Example FPGA Bitstreams 6.10. Preparing a ResNet50 v1 Model 6.11. Performing Inference on the Inflated 3D (I3D) Graph 6.12. Performing Inference on YOLOv3 and Calculating Accuracy Metrics 6.13. Performing Inference Without an FPGA Board
6.12. Performing Inference on YOLOv3 and Calculating Accuracy Metrics x
6.12.1. Preparing a YOLOv3 Model 6.12.2. Preparing a COCO Validation Dataset and Annotations 6.12.3. Inference on YOLOv3 and Calculating Accuracy Scores
1. FPGA AI Suite Getting Started Guide
2. FPGA AI Suite Components
3. FPGA AI Suite Installation Overview
4. Installing the FPGA AI Suite Compiler and IP Generation Tools
5. Installing the FPGA AI Suite PCIe-Based Design Example Prerequisites
6. FPGA AI Suite Quick Start Tutorial
7. Running the Hostless DDR-Free Design Example
A. FPGA AI Suite Getting Started Guide Archives
B. FPGA AI Suite Getting Started Guide Document Revision History

6.7. Performing Inference on the PCIe-Based Example Design

Performing Inference Using JIT Mode

The JIT (just-in-time) mode causes the dla_benchmark demonstration application to call the dla_compiler command in a just-in-time way to compile the neural net graph.

If you do not have images and ground truth files, you can skip the optional -i and -groundtruth_loc parameter entries in the command that follows. If you skip these parameters, the dla_benchmark demonstration application generates randomized image data.

The value for $curarch must match the bitstream that you programmed in Programming the FPGA Device. 

imagedir=$COREDLA_WORK/demo/sample_images
xmldir=$COREDLA_WORK/demo/models/public/ 
$COREDLA_WORK/runtime/build_Release/dla_benchmark/dla_benchmark \
    -b=1 \
    -m $xmldir/resnet-50-tf/FP32/resnet-50-tf.xml \
    -d=HETERO:FPGA,CPU \
    -niter=8 \
    -plugins $COREDLA_WORK/runtime/plugins.xml \
    -arch_file $curarch \
    -api=async \
    -perf_est \
    -nireq=4 \
    -bgr \
    -i $imagedir \
    -groundtruth_loc $imagedir/TF_ground_truth.txt

Performing Inference Using AOT Mode

In AOT (ahead-of-time) mode, the dla_benchmark demonstration application uses a compiled network that was previously produced by the dla_compiler compiler command when you followed the steps in Running the Graph Compiler.

To use AOT mode instead of JIT mode:

  1. Add the -cm argument to specify the name of the file containing the compiled network
  2. Remove the -perf_est flag. The dla_benchmark demonstration application does not estimate performance in AOT mode.

If you omit -i and -groundtruth_loc arguments, the dla_benchmark demonstration application generates random input data that is useful only for performance benchmarking. 

gt_file=$COREDLA_WORK/demo/sample_images/TF_ground_truth.txt
$COREDLA_WORK/runtime/build_Release/dla_benchmark/dla_benchmark \
    -b=1 \
    -cm $COREDLA_WORK/demo/RN50_Performance_b1.bin \
    -d=HETERO:FPGA,CPU \
    -niter=8 \
    -plugins $COREDLA_WORK/runtime/plugins.xml \
    -arch_file $curarch \
    -api=async \
    -nireq=4 \
    -bgr \
    -i $COREDLA_WORK/demo/sample_images/ \
    -groundtruth_loc $gt_file

The -cm argument points to the .bin file that you created in Running the Graph Compiler.

Inference APIs

The easiest way to evaluate the ability of the FPGA AI Suite to perform inference is to use the dla_benchmark demonstration application that is included in the example runtime and is built as part of the steps described in Programming the FPGA Device.

The example runtime also includes instructions on how to use the OpenVINO™ Python API to execute inference using the JIT style described in Performing Inference Using JIT Mode.

These instructions are located in $COREDLA_WORK/runtime/python_demos/README.md.

Level Two Title