Intel® FPGA AI Suite: SoC Design Example User Guide

Download PDF
ID 768979
Date 9/06/2023
Public

A newer version of this document is available. Customers should click here to go to the newest version.

Document Table of Contents
Document Table of Contents x
1. Intel® FPGA AI Suite SoC Design Example User Guide 2. About the SoC Design Example 3. Intel® FPGA AI Suite SoC Design Example Quick Start Tutorial 4. Intel® FPGA AI Suite SoC Design Example Run Process 5. Intel® FPGA AI Suite SoC Design Example Build Process 6. Intel® FPGA AI Suite SoC Design Example Intel® Quartus® Prime System Architecture 7. Intel® FPGA AI Suite Soc Design Example Software Components 8. Streaming-to-Memory (S2M) Streaming Demonstration A. Intel® FPGA AI Suite SoC Design Example User Guide Archives B. Intel® FPGA AI Suite SoC Design Example User Guide Document Revision History
2. About the SoC Design Example x
2.1. Intel® FPGA AI Suite SoC Design Example Prerequisites
3. Intel® FPGA AI Suite SoC Design Example Quick Start Tutorial x
3.1. Initial Setup 3.2. Initializing a Work Directory 3.3. (Optional) Create an SD Card Image (.wic) 3.4. Writing the SD Card Image (.wic) to an SD Card 3.5. Preparing the Intel® Arria® 10 SX SoC FPGA Development Kit for the Intel® FPGA AI Suite SoC Design Example 3.6. Adding Compiled Graphs (AOT files) to the SD Card 3.7. Running the Demonstration Applications
3.3. (Optional) Create an SD Card Image (.wic) x
3.3.1. Installing Prerequisite Software for Building an SD Card Image 3.3.2. Building the FPGA Bitstreams 3.3.3. Installing HPS Disk Image Build Prerequisites 3.3.4. (Optional) Downloading the ImageNet Categories 3.3.5. Building the SD Card Image
3.5. Preparing the Intel® Arria® 10 SX SoC FPGA Development Kit for the Intel® FPGA AI Suite SoC Design Example x
3.5.1. Confirming Intel® Arria® 10 SX SoC FPGA Development Kit Board Settings 3.5.2. Connect the Intel® Arria® 10 SX SoC FPGA Development Kit to the Host Development System 3.5.3. Configuring the Intel® Arria® 10 SX SoC FPGA Development Kit UART Connection 3.5.4. Determining the Intel® Arria® 10 SX SoC FPGA Development Kit IP Address
3.6. Adding Compiled Graphs (AOT files) to the SD Card x
3.6.1. Preparing OpenVINO™ Model Zoo 3.6.2. Preparing a Model 3.6.3. Compiling the Graphs 3.6.4. Copying the Compiled Graphs to the SD card
3.7. Running the Demonstration Applications x
3.7.1. Running the M2M Mode Demonstration Application 3.7.2. Running the S2M Mode Demonstration Application 3.7.3. Troubleshooting the Demonstration Applications
4. Intel® FPGA AI Suite SoC Design Example Run Process x
4.1. Exporting Trained Graphs from Source Frameworks 4.2. Compiling Exported Graphs Through the Intel® FPGA AI Suite
5. Intel® FPGA AI Suite SoC Design Example Build Process x
5.1. Building the Intel® Quartus® Prime Project 5.2. Building the Bootable SD Card Image (.wic)
5.1. Building the Intel® Quartus® Prime Project x
5.1.1. Intel® Quartus® Prime Build Flow 5.1.2. Build Script Options 5.1.3. Build Directory
5.1.1. Intel® Quartus® Prime Build Flow x
5.1.1.1. Build Synchronization of FPGA with Software
5.1.3. Build Directory x
5.1.3.1. The create_project.bash Script 5.1.3.2. The generate_sof.bash Script 5.1.3.3. The generate_rbf.bash Script 5.1.3.4. The build_stream_controller.sh Script
6. Intel® FPGA AI Suite SoC Design Example Intel® Quartus® Prime System Architecture x
6.1. Intel® FPGA AI Suite SoC Design Example Inference Sequence Overview 6.2. Memory-to-Memory (M2M) Variant Design 6.3. Streaming-to-Memory (S2M) Variant Design 6.4. Top Level 6.5. The SoC Design Example Platform Designer System 6.6. Fabric EMIF Design Component 6.7. PLL Configuration
6.2. Memory-to-Memory (M2M) Variant Design x
6.2.1. The mSGDMA Intel FPGA IP 6.2.2. RAM considerations
6.3. Streaming-to-Memory (S2M) Variant Design x
6.3.1. Streaming Enablement for Intel® FPGA AI Suite 6.3.2. Nios® V Subsystem 6.3.3. Streaming System Operation 6.3.4. Resolving Input Rate Mismatches Between the Intel® FPGA AI Suite IP and the Streaming Input 6.3.5. The Layout Transform IP as an Application-Specific Block
6.3.3. Streaming System Operation x
6.3.3.1. Streaming System Buffer Management 6.3.3.2. Streaming System Inference Job Management
6.3.5. The Layout Transform IP as an Application-Specific Block x
6.3.5.1. Layout Transform Considerations 6.3.5.2. Layout Transform IP Register Map 6.3.5.3. Layout Transform Configuration Options
6.4. Top Level x
6.4.1. Clock Domains
6.5. The SoC Design Example Platform Designer System x
6.5.1. The dla_0 Platform Designer Layer (dla.qsys) 6.5.2. The hps_0 Platform Designer Layer (hps.qys)
7. Intel® FPGA AI Suite Soc Design Example Software Components x
7.1. Yocto Build and Runtime Linux Environment 7.2. Intel® FPGA AI Suite Runtime Plugin 7.3. Runtime Interaction with the MMD Layer 7.4. MMD Layer Hardware Interaction Library
7.1. Yocto Build and Runtime Linux Environment x
7.1.1. Yocto Recipe: recipes-core/images/coredla-image.bb 7.1.2. Yocto Recipe: recipes-bsp/u-boot/u-boot-socfpga_%.bbapend 7.1.3. Yocto Recipe: recipes-drivers/msgdma-userio/msgdma-userio.bb 7.1.4. Yocto Recipe: recipes-drivers/uio-devices/uio-devices.bb 7.1.5. Yocto Recipe: recipes-kernel/linux/linux-socfpga-lts_5.15.bbappend 7.1.6. Yocto Recipe: wic
7.4. MMD Layer Hardware Interaction Library x
7.4.1. MMD Layer Hardware Interaction Library Class mmd_device 7.4.2. MMD Layer Hardware Interaction Library Class uio_device 7.4.3. MMD Layer Hardware Interaction Library Class dma_device
8. Streaming-to-Memory (S2M) Streaming Demonstration x
8.1. Nios® Subsystem 8.2. Building the Stream Controller Module 8.3. Building the Streaming Demonstration Applications 8.4. Running the Streaming Demonstration
8.1. Nios® Subsystem x
8.1.1. Stream Controller Communication Protocol 8.1.2. Buffer Flow in Streaming Mode using Nios® V Software Scheduler
8.1.2. Buffer Flow in Streaming Mode using Nios® V Software Scheduler x
8.1.2.1. Review of M2M mode 8.1.2.2. External Streaming Mode Buffer Flow 8.1.2.3. Nios® V Stream Controller State Machine Buffer Flow
8.4. Running the Streaming Demonstration x
8.4.1. The streaming_inference_app Application 8.4.2. The image_streaming_app Application
1. Intel® FPGA AI Suite SoC Design Example User Guide
2. About the SoC Design Example
3. Intel® FPGA AI Suite SoC Design Example Quick Start Tutorial
4. Intel® FPGA AI Suite SoC Design Example Run Process
5. Intel® FPGA AI Suite SoC Design Example Build Process
6. Intel® FPGA AI Suite SoC Design Example Intel® Quartus® Prime System Architecture
7. Intel® FPGA AI Suite Soc Design Example Software Components
8. Streaming-to-Memory (S2M) Streaming Demonstration
A. Intel® FPGA AI Suite SoC Design Example User Guide Archives
B. Intel® FPGA AI Suite SoC Design Example User Guide Document Revision History

7. Intel® FPGA AI Suite Soc Design Example Software Components

The Intel® FPGA AI Suite SoC design example contains a software environment for the runtime flow.

The software environment for the Intel® Arria® 10 SX SoC FPGA Development Kit consists of the following components:
  • Yocto build and runtime Linux environment
  • Intel® Distribution of OpenVINO™ Toolkit Version 2022.3.1 LTS (Inference Engine, Heterogeneous plugin)
  • OpenVINO™ Arm* CPU plugin
  • Intel® FPGA AI Suite runtime plugin
  • MMD hardware library

The Intel® FPGA AI Suite SoC design example contains the source files, Makefiles, and scripts to cross compile all the software for the Intel® Arria® 10 SX SoC FPGA Development Kit. The Yocto SDK provides the cross compiler, and is the first component that must be built.

The machine learning network graph is compiled separately using the OpenVINO™ Model Optimizer and the Intel® FPGA AI Suite compiler (dla_compiler) command.. When you compile the graph for the Intel® FPGA AI Suite SoC design example, ensure that you specify the --foutput-format=open_vino_hetero and -o <path_to_file>/CompiledNetwork.bin options.

The AOT file from the Intel® FPGA AI Suite compiler contains the compiled network partitions for FPGA and CPU devices along with the network weights. The network is compiled for a specific Intel® FPGA AI Suite architecture and batch size.

The SoC flow does not support the Just-In-Time (JIT) flow because Arm* libraries are not available for the Intel® FPGA AI Suite compiler.

An architecture file (.arch) describes the Intel® FPGA AI Suite IP architecture to the compiler. You must specify the same architecture file to the Intel® FPGA AI Suite compiler and to the Intel® FPGA AI Suite design example build script (dla_build_example_design.py).

The runtime stack cannot program the FPGA with a bitstream. The bitstream must be built into the SD card (.wic) image that is used to program the flash card, as described in (Optional) Create an SD Card Image (.wic) and Writing the SD Card Image (.wic) to an SD Card.

The runtime inference on the Intel® Arria® 10 SX SoC uses the OpenVINO™ Arm* CPU plugin. To enable fallback to the OpenVINO™ Arm* CPU plugin for graph layers that are not supported on the FPGA, the device flag must be set to HETERO:FPGA,CPU during the AOT compile step and when you run the dla_benchmark command.

In some cases, a layer might be supported by the FPGA even though the OpenVINO™ Arm* CPU plugin does not support the layer. This support is handled by the HETERO plugin and the layer is executed on the FPGA as expected. As an example, 3D convolution layers are not supported by the OpenVINO™ Arm* CPU plugin but still work properly provided that the .arch file used for the Intel® FPGA AI Suite IP configuration has enabled support for 3D convolutions.


Section Content
Yocto Build and Runtime Linux Environment
Intel FPGA AI Suite Runtime Plugin
Runtime Interaction with the MMD Layer
MMD Layer Hardware Interaction Library

Related Information
Level Two Title