FPGA AI Suite: PCIe-based Design Example User Guide

ID 768977
Date 7/31/2024
Public
Document Table of Contents

6.3. Software Components

The PCIe* -based design example contains a sample software stack for the runtime flow.

The following figure, Software Stacks for FPGA AI Suite Inference, shows the complete runtime stack.

For the Agilex™ 7 design example, the following components comprise the runtime stack:
  • OpenVINO™ Toolkit 2023.3 LTS (Inference Engine, Heterogeneous Plugin)
  • FPGA AI Suite runtime plugin
  • Terasic DE10-Agilex-B2E2 board driver

The PCIe* -based design example contains the source files and Makefiles to build the FPGA AI Suite runtime plugin. The OpenVINO™ component is external and must be manually pre-installed.

A separate flow compiles the AI network graph using the FPGA AI Suite compiler, as shown in figure Software Stacks for FPGA AI Suite Inference below as the Compilation Software Stack.

The compilation flow output is a single binary file called CompiledNetwork.bin that contains the compiled network partitions for FPGA and CPU devices along with the network weights. The network is compiled for a specific FPGA AI Suite architecture and batch size. This binary is created on-disk only when using the Ahead-Of-Time flow; when the JIT flow is used, the compiled object stays in-memory only.

An Architecture File describes the FPGA AI Suite IP architecture to the compiler. You must specify the same Architecture File to the FPGA AI Suite compiler and to the FPGA AI Suite PCIe Example Design build script (dla_build_example_design.py).

The runtime flow accepts the CompiledNetwork.bin file as the input network along with the image data files.

Figure 1. Software Stacks for FPGA AI Suite Inference

The runtime stack cannot program the FPGA with a bitstream. To build a bitstream and program the FPGA devices:

  1. Compile the design example. For details, refer to Compiling the PCIe* -based Design Example.
  2. Program the device with the bitstream. For details, refer to Programming the FPGA Device ( Agilex 7).

To run inference through the OpenVINO™ Toolkit on the FPGA, set the OpenVINO™ device configuration flag (used by the heterogeneous Plugin) to FPGA or HETERO:FPGA,CPU.