FPGA AI Suite: IP Reference Manual

Download PDF
ID 768974
Date 9/06/2024
Public
Document Table of Contents
Document Table of Contents x
1. FPGA AI Suite IP Reference Manual 2. About the FPGA AI Suite IP 3. FPGA AI Suite IP Generation Utility 4. FPGA AI Suite Ahead-of-Time Splitter Utility 5. CSR Map and Descriptor Queue A. FPGA AI Suite IP Reference Manual Archives B. FPGA AI Suite IP Reference Manual Document Revision History
2. About the FPGA AI Suite IP x
2.1. Supported Models 2.2. Model Performance 2.3. Software Emulation of the FPGA AI Suite IP 2.4. FPGA AI Suite Layer / Primitive Ranges 2.5. FPGA AI Suite IP Block Configuration 2.6. IP Block Interfaces 2.7. Feature Input and Output Streaming 2.8. DDR-Free Operation
2.1. Supported Models x
2.1.1. MobileNet V2 differences between Caffe and TensorFlow models
2.2. Model Performance x
2.2.1. Throughput on the MobileNetV1 model (and other very fast models)
2.5. FPGA AI Suite IP Block Configuration x
2.5.1. Architecture Description File Format for Instance Parameterization 2.5.2. Architecture Description File Parameters
2.5.2. Architecture Description File Parameters x
2.5.2.1. Parameter Group: Global Parameters 2.5.2.2. Parameter Group: activation 2.5.2.3. Parameter Group: pe_array 2.5.2.4. Parameter Group: pool 2.5.2.5. Parameter Group: depthwise 2.5.2.6. Module: softmax 2.5.2.7. Parameter Group: dma 2.5.2.8. Parameter Group: xbar 2.5.2.9. Parameter Group: filter_scratchpad 2.5.2.10. Parameter Group: input_stream_interface 2.5.2.11. Parameter Group: output_stream_interface 2.5.2.12. Parameter Group: config_network 2.5.2.13. (Early Access) Parameter Group: layout_transform_params
2.6. IP Block Interfaces x
2.6.1. Clock and Reset 2.6.2. AXI Interfaces 2.6.3. AXI Interface Clock and Reset 2.6.4. Input Feature Tensor In-Memory Format 2.6.5. Output Tensor In-Memory Format
2.6.4. Input Feature Tensor In-Memory Format x
2.6.4.1. Multiple Input Graphs 2.6.4.2. Input Folding 2.6.4.3. Input Scale and Shift 2.6.4.4. Input Transform Mapping 2.6.4.5. Input Layout Transform Hardware
2.7. Feature Input and Output Streaming x
2.7.1. Input Streaming 2.7.2. Output Streaming
3. FPGA AI Suite IP Generation Utility x
3.1. IP Generation Utility Execution Flows 3.2. IP Generation Utility Inputs 3.3. IP Generation Utility Outputs 3.4. IP Generation Utility Command Line Options
3.4. IP Generation Utility Command Line Options x
3.4.1. The --flow create_ip Flow 3.4.2. The --flow add_arch Flow 3.4.3. The --flow list Flow 3.4.4. The --flow remove_arch Flow
4. FPGA AI Suite Ahead-of-Time Splitter Utility x
4.1. Files Generated by the FPGA AI Suite Ahead-of-Time (AOT) Splitter Utility 4.2. Building the FPGA AI Suite Ahead-of-Time (AOT) Splitter Utility 4.3. Running the FPGA AI Suite Ahead-of-Time (AOT) Splitter Utility 4.4. FPGA AI Suite Ahead-of-Time (AOT) Splitter Utility Example Application
5. CSR Map and Descriptor Queue x
5.1. Discovery ROM 5.2. Interrupt Control 5.3. DMA Descriptor Queue 5.4. DMA Control Registers 5.5. Performance Registers 5.6. Debug Network Registers 5.7. DMA License Register
1. FPGA AI Suite IP Reference Manual
2. About the FPGA AI Suite IP
3. FPGA AI Suite IP Generation Utility
4. FPGA AI Suite Ahead-of-Time Splitter Utility
5. CSR Map and Descriptor Queue
A. FPGA AI Suite IP Reference Manual Archives
B. FPGA AI Suite IP Reference Manual Document Revision History

2.8. Memoryless Operation

To avoid use of external memory for storing graph weights and FPGA AI Suite IP configurations during inference, you can store these parameters within the IP using on-chip memory.

In this case, three types of memory initialization files (.mif) are required:

  • memoryless_filter_hw*.mif: Contains the graph filters.
  • memoryless_bias_scale_hw*.mif: Contains the graph biases and scaling factors.
  • memoryless_config.mif: Stores the FPGA AI Suite instructions for the compiled graph.

These files are generated using the dla_compiler tool that takes the architecture definition (.arch file) and the target neural network graph as inputs.

A total of K_VECTOR memoryless filter .mif files are generated. Each file is postfixed with an integer representing which PE the filter is to be loaded to.

Similarly, a total of K_VECTOR memoryless bias and scale files are generated, using the same postfix scheme to indicate the PE that should load this file.

The FPGA AI Suite IP requires a stream of instructions that describe the order in which convolutions, activations, and other operations must be performed. This instruction stream is stored in a single .mif file (memoryless_config.mif).

These files are used to initialize read-only memories while building an FPGA bitstream, which means that bitstreams are graph-specific for memoryless operation.

You must also enable input and output streaming when using memoryless operation. The graph must have a large enough stream buffer depth to accommodate all of the intermediate results during inference.

The dla_build_example_design.py command can build design examples with memoryless operation. The only streaming design example option currently supported is the 0_STREAMING option. The design example created by this option targets an Agilex™ 7 FPGA I-Series Development Kit (DK-DEV-AGI027RBES).

To compile a bitstream with a memoryless architecture, specify the directory that contains the memoryless .mif files with the –parameter_rom_dir option of the dla_build_example_design.py command.

For details about creating the .mif file required for memoryless operation, refer to "Generating Artifacts for Memoryless Operation" in the FPGA AI Suite Compiler Reference Manual .

Level Two Title