FPGA AI Suite: PCIe-based Design Example User Guide

Download PDF
ID 768977
Date 7/31/2024
Public
Document Table of Contents
Document Table of Contents x
1. FPGA AI Suite PCIe-based Design Example User Guide 2. About the PCIe* -based Design Example 3. Getting Started with the FPGA AI Suite PCIe* -based Design Example 4. Building the FPGA AI Suite Runtime 5. Running the Design Example Demonstration Applications 6. Design Example Components 7. Design Example System Architecture for the Agilex™ 7 FPGA A. FPGA AI Suite PCIe-based Design Example User Guide Archives B. FPGA AI Suite PCIe-based Design Example User Guide Document Revision History
4. Building the FPGA AI Suite Runtime x
4.1. CMake Targets 4.2. Build Options
5. Running the Design Example Demonstration Applications x
5.1. Exporting Trained Graphs from Source Frameworks 5.2. Compiling Exported Graphs Through the FPGA AI Suite 5.3. Compiling the PCIe* -based Example Design 5.4. Programming the FPGA Device ( Agilex™ 7) 5.5. Performing Accelerated Inference with the dla_benchmark Application 5.6. Running the Ported OpenVINO™ Demonstration Applications
5.5. Performing Accelerated Inference with the dla_benchmark Application x
5.5.1. Inference on Image Classification Graphs 5.5.2. Inference on Object Detection Graphs 5.5.3. Additional dla_benchmark Options 5.5.4. The dla_benchmark Performance Metrics
5.5.2. Inference on Object Detection Graphs x
5.5.2.1. The mAP and COCO AP Metrics 5.5.2.2. Specifying Ground Truth 5.5.2.3. Example of Inference on Object Detection Graphs
5.5.4. The dla_benchmark Performance Metrics x
5.5.4.1. Interpreting System Throughput and Latency Metrics
5.6. Running the Ported OpenVINO™ Demonstration Applications x
5.6.1. Example Running the Object Detection Demonstration Application
6. Design Example Components x
6.1. Build Script 6.2. Example Architecture Bitstream Files 6.3. Software Components 6.4. Software Interface to the BSP
6.1. Build Script x
6.1.1. Build Script Options 6.1.2. Script Flow
6.3. Software Components x
6.3.1. OpenVINO™ FPGA Runtime Plugin 6.3.2. FPGA AI Suite Runtime 6.3.3. BSP Driver
7. Design Example System Architecture for the Agilex™ 7 FPGA x
7.1. System Overview 7.2. Hardware
7.2. Hardware x
7.2.1. PLL Adjustment
1. FPGA AI Suite PCIe-based Design Example User Guide
2. About the PCIe* -based Design Example
3. Getting Started with the FPGA AI Suite PCIe* -based Design Example
4. Building the FPGA AI Suite Runtime
5. Running the Design Example Demonstration Applications
6. Design Example Components
7. Design Example System Architecture for the Agilex™ 7 FPGA
A. FPGA AI Suite PCIe-based Design Example User Guide Archives
B. FPGA AI Suite PCIe-based Design Example User Guide Document Revision History

6.3.1. OpenVINO™ FPGA Runtime Plugin

The FPGA runtime plugin uses the OpenVINO™ Inference Engine Plugin API.

The OpenVINO™ Plugin architecture is described in the OpenVINO™ Developer Guide for Inference Engine Plugin Library.

The source files are located under runtime/plugin. The three main components of the runtime plugin are the Plugin class, the Executable Network class, and the Inference Request class. The primary responsibilities for each class are as follows:

Plugin class

  • Initializes the runtime plugin with an FPGA AI Suite architecture file which you set as an OpenVINO™ configuration key (refer to Running the Ported OpenVINO Demonstration Applications).
  • Contains QueryNetwork function that analyzes network layers and returns a list of layers that the specified architecture supports. This function allows network execution to be distributed between FPGA and other devices and is enabled with the HETERO mode.
  • Creates an executable network instance in one of the following ways:
    • Just-in-time (JIT) flow: Compiles a network such that the compiled network is compatible with the hardware corresponding to the FPGA AI Suite architecture file, and then loads the compiled network onto the FPGA device.
    • Ahead-of-time (AOT) flow: Imports a precompiled network (exported by FPGA AI Suite compiler) and loads it onto the FPGA device.

Executable Network Class

  • Represents an FPGA AI Suite compiled network
  • Loads the compiled model and config data for the network onto the FPGA device that has already been programmed with an FPGA AI Suite bitstream. For two instances of FPGA AI Suite, the Executable Network class loads the network onto both instances, allowing them to perform parallel batch inference.
  • Stores input/output processing information.
  • Creates infer request instances for pipelining multiple batch execution.

Infer Request class

  • Runs a single batch inference serially.
  • Executes five stages in one inference job – input layout transformation on CPU, input transfer to DDR, FPGA AI Suite FPGA execution, output transfer from DDR, output layout transformation on CPU.
  • In asynchronous mode, executes the stages on multiple threads that are shared across all inference request instances so that multiple batch jobs are pipelined, and the FPGA is always active.
Related Information
Level Two Title