FPGA AI Suite: SoC Design Example User Guide

Download PDF
ID 768979
Date 7/31/2024
Public
Document Table of Contents
Document Table of Contents x
1. FPGA AI Suite SoC Design Example User Guide 2. About the SoC Design Example 3. FPGA AI Suite SoC Design Example Quick Start Tutorial 4. FPGA AI Suite SoC Design Example Run Process 5. FPGA AI Suite SoC Design Example Build Process 6. FPGA AI Suite SoC Design Example Quartus® Prime System Architecture 7. FPGA AI Suite Soc Design Example Software Components 8. Streaming-to-Memory (S2M) Streaming Demonstration A. FPGA AI Suite SoC Design Example User Guide Archives B. FPGA AI Suite SoC Design Example User Guide Document Revision History
2. About the SoC Design Example x
2.1. FPGA AI Suite SoC Design Example Prerequisites
2.1. FPGA AI Suite SoC Design Example Prerequisites x
2.1.1. Agilex™ 7 FPGA I-Series Transceiver-SoC Development Kit Hardware Requirements
3. FPGA AI Suite SoC Design Example Quick Start Tutorial x
3.1. Initial Setup 3.2. Initializing a Work Directory 3.3. (Optional) Create an SD Card Image (.wic) 3.4. Writing the SD Card Image (.wic) to an SD Card 3.5. Preparing SoC FPGA Development Kits for the FPGA AI Suite SoC Design Example 3.6. Adding Compiled Graphs (AOT files) to the SD Card 3.7. Verifying FPGA Device Drivers 3.8. Running the Demonstration Applications
3.3. (Optional) Create an SD Card Image (.wic) x
3.3.1. Installing Prerequisite Software for Building an SD Card Image 3.3.2. Building the FPGA Bitstreams 3.3.3. Installing HPS Disk Image Build Prerequisites 3.3.4. (Optional) Downloading the ImageNet Categories 3.3.5. Building the SD Card Image
3.5. Preparing SoC FPGA Development Kits for the FPGA AI Suite SoC Design Example x
3.5.1. Preparing the Agilex™ 7 FPGA I-Series Transceiver-SoC Development Kit 3.5.2. Preparing the Arria® 10 SX SoC FPGA Development Kit 3.5.3. Configuring the SoC FPGA Development Kit UART Connection 3.5.4. Determining the SoC FPGA Development Kit IP Address
3.5.1. Preparing the Agilex™ 7 FPGA I-Series Transceiver-SoC Development Kit x
3.5.1.1. Confirming Agilex™ 7 FPGA I-Series Transceiver-SoC Development Kit Board Set Up 3.5.1.2. Programming the Agilex™ 7FPGA Device with the JTAG Indirect Configuration (.jic) File 3.5.1.3. Connecting the Agilex™ 7 FPGA I-Series Transceiver-SoC Development Kit to the Host Development System
3.5.2. Preparing the Arria® 10 SX SoC FPGA Development Kit x
3.5.2.1. Confirming Arria® 10 SX SoC FPGA Development Kit Board Settings 3.5.2.2. Connecting the Arria® 10 SX SoC FPGA Development Kit to the Host Development System
3.6. Adding Compiled Graphs (AOT files) to the SD Card x
3.6.1. Preparing OpenVINO™ Model Zoo 3.6.2. Preparing a Model 3.6.3. Compiling the Graphs 3.6.4. Copying the Compiled Graphs to the SD card
3.8. Running the Demonstration Applications x
3.8.1. Running the M2M Mode Demonstration Application 3.8.2. Running the S2M Mode Demonstration Application 3.8.3. Troubleshooting the Demonstration Applications
4. FPGA AI Suite SoC Design Example Run Process x
4.1. Exporting Trained Graphs from Source Frameworks 4.2. Compiling Exported Graphs Through the FPGA AI Suite
5. FPGA AI Suite SoC Design Example Build Process x
5.1. Building the Quartus® Prime Project 5.2. Building the Bootable SD Card Image (.wic)
5.1. Building the Quartus® Prime Project x
5.1.1. Quartus® Prime Build Flow 5.1.2. Build Script Options 5.1.3. Build Directory
5.1.1. Quartus® Prime Build Flow x
5.1.1.1. Build Synchronization of FPGA with Software
5.1.3. Build Directory x
5.1.3.1. The create_project.bash Script 5.1.3.2. The generate_sof.bash Script 5.1.3.3. The generate_rbf.bash Script 5.1.3.4. The build_stream_controller.sh Script
6. FPGA AI Suite SoC Design Example Quartus® Prime System Architecture x
6.1. FPGA AI Suite SoC Design Example Inference Sequence Overview 6.2. Memory-to-Memory (M2M) Variant Design 6.3. Streaming-to-Memory (S2M) Variant Design 6.4. Top Level 6.5. The SoC Design Example Platform Designer System 6.6. Fabric EMIF Design Component 6.7. PLL Configuration
6.2. Memory-to-Memory (M2M) Variant Design x
6.2.1. The mSGDMA Intel FPGA IP 6.2.2. RAM considerations
6.3. Streaming-to-Memory (S2M) Variant Design x
6.3.1. Streaming Enablement for FPGA AI Suite 6.3.2. Nios® V Subsystem 6.3.3. Streaming System Operation 6.3.4. Resolving Input Rate Mismatches Between the FPGA AI Suite IP and the Streaming Input 6.3.5. The Layout Transform IP as an Application-Specific Block
6.3.3. Streaming System Operation x
6.3.3.1. Streaming System Buffer Management 6.3.3.2. Streaming System Inference Job Management
6.3.5. The Layout Transform IP as an Application-Specific Block x
6.3.5.1. Layout Transform Considerations 6.3.5.2. Layout Transform IP Register Map 6.3.5.3. Layout Transform Configuration Options
6.4. Top Level x
6.4.1. Clock Domains
6.5. The SoC Design Example Platform Designer System x
6.5.1. The dla_0 Platform Designer Layer (dla.qsys) 6.5.2. The hps_0 Platform Designer Layer (hps.qys)
7. FPGA AI Suite Soc Design Example Software Components x
7.1. Yocto Build and Runtime Linux Environment 7.2. FPGA AI Suite Runtime Plugin 7.3. Runtime Interaction with the MMD Layer 7.4. MMD Layer Hardware Interaction Library
7.1. Yocto Build and Runtime Linux Environment x
7.1.1. Yocto Recipe: recipes-core/images/coredla-image.bb 7.1.2. Yocto Recipe: recipes-bsp/u-boot/u-boot-socfpga_%.bbappend 7.1.3. Yocto Recipe: recipes-drivers/msgdma-userio/msgdma-userio.bb 7.1.4. Yocto Recipe: recipes-drivers/uio-devices/uio-devices.bb 7.1.5. Yocto Recipe: recipes-kernel/linux/linux-socfpga-lts_5.15.bbappend 7.1.6. Yocto Recipe: wic
7.4. MMD Layer Hardware Interaction Library x
7.4.1. MMD Layer Hardware Interaction Library Class mmd_device 7.4.2. MMD Layer Hardware Interaction Library Class uio_device 7.4.3. MMD Layer Hardware Interaction Library Class dma_device
8. Streaming-to-Memory (S2M) Streaming Demonstration x
8.1. Nios® Subsystem 8.2. Building the Stream Controller Module 8.3. Building the Streaming Demonstration Applications 8.4. Running the Streaming Demonstration
8.1. Nios® Subsystem x
8.1.1. Stream Controller Communication Protocol 8.1.2. Buffer Flow in Streaming Mode using Nios® V Software Scheduler
8.1.2. Buffer Flow in Streaming Mode using Nios® V Software Scheduler x
8.1.2.1. Review of M2M mode 8.1.2.2. External Streaming Mode Buffer Flow 8.1.2.3. Nios® V Stream Controller State Machine Buffer Flow
8.4. Running the Streaming Demonstration x
8.4.1. The streaming_inference_app Application 8.4.2. The image_streaming_app Application
1. FPGA AI Suite SoC Design Example User Guide
2. About the SoC Design Example
3. FPGA AI Suite SoC Design Example Quick Start Tutorial
4. FPGA AI Suite SoC Design Example Run Process
5. FPGA AI Suite SoC Design Example Build Process
6. FPGA AI Suite SoC Design Example Quartus® Prime System Architecture
7. FPGA AI Suite Soc Design Example Software Components
8. Streaming-to-Memory (S2M) Streaming Demonstration
A. FPGA AI Suite SoC Design Example User Guide Archives
B. FPGA AI Suite SoC Design Example User Guide Document Revision History

8.4.2. The image_streaming_app Application

The image_streaming_app application loads images from the SD card, programs the layout transform IP, and then transfers the buffers via a streaming mSGDMA interface to the device. The buffers are sent at regular intervals at a frequency set by one of the command line options. Buffers are continually sent until the program is stopped with Ctrl+C.

The image_streaming_app application works only with .bmp files with dimensions of 224x224 pixels. The .bmp files can be either 24 or 32 bits per pixel format. If they are 24 bits, the buffers are padded to make them 32 bits per pixel. This format is expected by the input of the layout transform IP .

The command usage from the -help command option is as follows: 
root@agilex7:~/app# ./image_streaming_app --help
Usage:
 image_streaming_app [Options]

Options:
-images_folder=folder     Location of bitmap files. Defaults to working folder.
-image=path               Location of a single bitmap file for single inference.
-send=n                   Number of images to stream. Default is 1 if -image is set, otherwise infinite.
-rate=n                   Rate to stream images, in Hz. n is an integer. Default is 30.
-width=n                  Image width in pixels, default = 224
-height=n                 Image height in pixels, default = 224
-c_vector=n               C vector size, default = 32
-blue_variance=n          Blue variance, default = 1.0
-green_variance=n         Green variance, default = 1.0
-red_variance=n           Red variance, default = 1.0
-blue_shift=n             Blue shift, default = -103.94
-green_shift=n            Green shift, default -116.78
-red_shift=n              Red shift, default = -123.68

The distribution includes a shell script utility called run_image_stream.sh that calls the image_streaming_app command with a rate of 50 Hz and default layout transform settings.

Once both applications are running, the streaming_inference_app application outputs the results to the terminal and a results.txt file.

Example output from the streaming_inference_app application is as follows: 
root@agilex7:~/app# ./run_inference_stream.sh
Runtime version check is enabled.
[ INFO ] Architecture used to compile the imported model: AGX7_Performance
Using licensed IP
Read hash from bitstream ROM...
Read build version string from bitstream ROM...
Read arch name string from bitstream ROM...
Runtime arch check is enabled. Check started...
Runtime arch check passed.
Runtime build version check is enabled. Check started...
Runtime build version check passed.
Ready to start image input stream.
1 - class ID 776, score = 58.4
2 - class ID 968, score = 90.7
3 - class ID 769, score = 97.8
4 - class ID 769, score = 97.8
5 - class ID 872, score = 99.8
6 - class ID 954, score = 94.4
7 - class ID 954, score = 94.4
8 - class ID 776, score = 58.4
9 - class ID 872, score = 99.8
10 - class ID 968, score = 90.7
11 - class ID 776, score = 58.4
12 - class ID 968, score = 90.7
13 - class ID 769, score = 97.8
14 - class ID 769, score = 97.8
15 - class ID 872, score = 99.8
16 - class ID 954, score = 94.4
17 - class ID 954, score = 94.4
18 - class ID 776, score = 58.4
19 - class ID 872, score = 99.8
20 - class ID 968, score = 90.7
^C
Ctrl+C detected. Shutting down application
The resulting results.txt file from the example is as follows: 
root@agilex7:~/app# cat results.txt
Result: image[1]
1. class ID 776, score = 58.4
2. class ID 683, score = 27.6
3. class ID 513, score = 10.2
4. class ID 432, score = 1.8
5. class ID 558, score = 1.6

Result: image[2]
1. class ID 968, score = 90.7
2. class ID 901, score = 1.1
3. class ID 868, score = 1.0
4. class ID 899, score = 1.0
5. class ID 725, score = 0.9

Result: image[3]
1. class ID 769, score = 97.8
2. class ID 845, score = 0.5
3. class ID 587, score = 0.4
4. class ID 798, score = 0.1
5. class ID 618, score = 0.1

Result: image[4]
1. class ID 769, score = 97.8
2. class ID 845, score = 0.5
3. class ID 587, score = 0.4
4. class ID 798, score = 0.1
5. class ID 618, score = 0.1
Level Two Title