FPGA AI Suite: IP Reference Manual

Download PDF
ID 768974
Date 9/06/2024
Public
Document Table of Contents
Document Table of Contents x
1. FPGA AI Suite IP Reference Manual 2. About the FPGA AI Suite IP 3. FPGA AI Suite IP Generation Utility 4. FPGA AI Suite Ahead-of-Time Splitter Utility 5. CSR Map and Descriptor Queue A. FPGA AI Suite IP Reference Manual Archives B. FPGA AI Suite IP Reference Manual Document Revision History
2. About the FPGA AI Suite IP x
2.1. Supported Models 2.2. Model Performance 2.3. Software Emulation of the FPGA AI Suite IP 2.4. FPGA AI Suite Layer / Primitive Ranges 2.5. FPGA AI Suite IP Block Configuration 2.6. IP Block Interfaces 2.7. Feature Input and Output Streaming 2.8. DDR-Free Operation
2.1. Supported Models x
2.1.1. MobileNet V2 differences between Caffe and TensorFlow models
2.2. Model Performance x
2.2.1. Throughput on the MobileNetV1 model (and other very fast models)
2.5. FPGA AI Suite IP Block Configuration x
2.5.1. Architecture Description File Format for Instance Parameterization 2.5.2. Architecture Description File Parameters
2.5.2. Architecture Description File Parameters x
2.5.2.1. Parameter Group: Global Parameters 2.5.2.2. Parameter Group: activation 2.5.2.3. Parameter Group: pe_array 2.5.2.4. Parameter Group: pool 2.5.2.5. Parameter Group: depthwise 2.5.2.6. Module: softmax 2.5.2.7. Parameter Group: dma 2.5.2.8. Parameter Group: xbar 2.5.2.9. Parameter Group: filter_scratchpad 2.5.2.10. Parameter Group: input_stream_interface 2.5.2.11. Parameter Group: output_stream_interface 2.5.2.12. Parameter Group: config_network 2.5.2.13. (Early Access) Parameter Group: layout_transform_params
2.6. IP Block Interfaces x
2.6.1. Clock and Reset 2.6.2. AXI Interfaces 2.6.3. AXI Interface Clock and Reset 2.6.4. Input Feature Tensor In-Memory Format 2.6.5. Output Tensor In-Memory Format
2.6.4. Input Feature Tensor In-Memory Format x
2.6.4.1. Multiple Input Graphs 2.6.4.2. Input Folding 2.6.4.3. Input Scale and Shift 2.6.4.4. Input Transform Mapping 2.6.4.5. Input Layout Transform Hardware
2.7. Feature Input and Output Streaming x
2.7.1. Input Streaming 2.7.2. Output Streaming
3. FPGA AI Suite IP Generation Utility x
3.1. IP Generation Utility Execution Flows 3.2. IP Generation Utility Inputs 3.3. IP Generation Utility Outputs 3.4. IP Generation Utility Command Line Options
3.4. IP Generation Utility Command Line Options x
3.4.1. The --flow create_ip Flow 3.4.2. The --flow add_arch Flow 3.4.3. The --flow list Flow 3.4.4. The --flow remove_arch Flow
4. FPGA AI Suite Ahead-of-Time Splitter Utility x
4.1. Files Generated by the FPGA AI Suite Ahead-of-Time (AOT) Splitter Utility 4.2. Building the FPGA AI Suite Ahead-of-Time (AOT) Splitter Utility 4.3. Running the FPGA AI Suite Ahead-of-Time (AOT) Splitter Utility 4.4. FPGA AI Suite Ahead-of-Time (AOT) Splitter Utility Example Application
5. CSR Map and Descriptor Queue x
5.1. Discovery ROM 5.2. Interrupt Control 5.3. DMA Descriptor Queue 5.4. DMA Control Registers 5.5. Performance Registers 5.6. Debug Network Registers 5.7. DMA License Register
1. FPGA AI Suite IP Reference Manual
2. About the FPGA AI Suite IP
3. FPGA AI Suite IP Generation Utility
4. FPGA AI Suite Ahead-of-Time Splitter Utility
5. CSR Map and Descriptor Queue
A. FPGA AI Suite IP Reference Manual Archives
B. FPGA AI Suite IP Reference Manual Document Revision History

2.6.4.2. Input Folding

Many graphs, particularly those processing image data, have very shallow input channels. In high CVEC instantiations, very shallow input channels can lead to low computational efficiency.

The folding can be done in conjunction with the FPGA AI Suite compiler and the FPGA AI Suite IP, or it can be performed in hardware using the hardware layout transform described in Input Layout Transform Hardware.

Input folding is typically most beneficial to the first layer of a graph.

The following figure illustrates an example of the functionality of the folding transform performed by the FPGA AI Suite compiler.

Figure 4. Illustration of the folding transform for a 1x1×5×5 input, 1×1×3×3 filter and stride_height = stride_width = 2

In this transformation, the input depth, height, and width are folded into the channel dimension by a factor corresponding to the stride of the first convolution of a network. In the earlier figure, this factor corresponds to transforming the input channels from 1 to 4 (), input height from 5 to 3 () and input width from 5 to 3. Each color corresponds to the new filter window, which in this case would be 4×1×2×2, with the gray boxes corresponding to 0 padding for the filters. Folding is done in a similar way for inputs with depths greater than one, but the folding transform illustration excludes it for simplicity.

The FPGA AI Suite IP has various enhancements that reduce, but not eliminate the efficiency hit of shallow first layers. In many cases, you can disable first layer folding in the compiler and pass shallow-channel tensors directly to the IP hardware.

You can disable or enable folding with the following FPGA AI Suite compiler options:

  • NoFolding

    No folding is performed by the host or an external module. This leads to low efficiency for the FPGA AI Suite IP but might be useful for debugging purposes.

  • ExternalFullFolding

    Folding performed by the host or an external module for the depth, height, and width stride of the first convolution layer.

  • ExternalFullExtraPEFolding

    Folding is performed by the host or an external module for the depth, height, and width stride of the first convolution layer with additional folding performed afterward via the FPGA AI Suite IP. This might lead to better performance than ExternalFullFolding depending on the instantiation parameters of IP.

  • PEFolding

    Folding is performed entirely by the FPGA AI Suite IP without the need for any host or external module. This should lead to similar performance to ExternalFullFolding depending on the instantiation parameters of the FPGA AI Suite IP and the neural network topology.

    PEFolding mode does not support input with a depth greater than one.

Level Two Title