Intel® FPGA AI Suite: IP Reference Manual

Download PDF
ID 768974
Date 9/06/2023
Public

A newer version of this document is available. Customers should click here to go to the newest version.

Document Table of Contents
Document Table of Contents x
1. Intel® FPGA AI Suite IP Reference Manual 2. About the Intel® FPGA AI Suite IP 3. Intel® FPGA AI Suite IP Generation Utility 4. Intel® FPGA AI Suite Ahead-of-Time Splitter Utility 5. CSR Map and Descriptor Queue A. Intel® FPGA AI Suite IP Reference Manual Archives B. Intel® FPGA AI Suite IP Reference Manual Document Revision History
2. About the Intel® FPGA AI Suite IP x
2.1. Supported Models 2.2. Model Performance 2.3. Intel® FPGA AI Suite Layer / Primitive Ranges 2.4. Intel® FPGA AI Suite IP Block Configuration 2.5. IP Block Interfaces
2.1. Supported Models x
2.1.1. MobileNet V2 differences between Caffe and TensorFlow models
2.2. Model Performance x
2.2.1. Throughput on the MobileNetV1 model (and other very fast models)
2.4. Intel® FPGA AI Suite IP Block Configuration x
2.4.1. Architecture Description File Format for Instance Parameterization 2.4.2. Architecture Description File Parameters
2.4.2. Architecture Description File Parameters x
2.4.2.1. Parameter group: Global Parameters 2.4.2.2. Parameter group: activation 2.4.2.3. Parameter group: pe_array 2.4.2.4. Parameter group: pool 2.4.2.5. Module: softmax 2.4.2.6. Parameter group: dma 2.4.2.7. Parameter group: xbar 2.4.2.8. Parameter group: filter_scratchpad 2.4.2.9. Parameter group: config_network
2.5. IP Block Interfaces x
2.5.1. Clock and Reset 2.5.2. AXI Interfaces 2.5.3. AXI Interface Clock and Reset 2.5.4. Input Feature Tensor In-Memory Format 2.5.5. Output Tensor In-Memory Format
2.5.4. Input Feature Tensor In-Memory Format x
2.5.4.1. Multiple Input Graphs 2.5.4.2. Input Folding 2.5.4.3. Input Scale and Shift 2.5.4.4. Input Transform Mapping
3. Intel® FPGA AI Suite IP Generation Utility x
3.1. IP Generation Utility Execution Flows 3.2. IP Generation Utility Inputs 3.3. IP Generation Utility Outputs 3.4. IP Generation Utility Command Line Options
3.4. IP Generation Utility Command Line Options x
3.4.1. The --flow create_ip Flow 3.4.2. The --flow add_arch Flow 3.4.3. The --flow list Flow 3.4.4. The --flow remove_arch Flow
4. Intel® FPGA AI Suite Ahead-of-Time Splitter Utility x
4.1. Files Generated by the Intel® FPGA AI Suite Ahead-of-Time (AOT) Splitter Utility 4.2. Building the Intel® FPGA AI Suite Ahead-of-Time (AOT) Splitter Utility 4.3. Running the Intel® FPGA AI Suite Ahead-of-Time (AOT) Splitter Utility 4.4. Intel® FPGA AI Suite Ahead-of-Time (AOT) Splitter Utility Example Application
5. CSR Map and Descriptor Queue x
5.1. Discovery ROM 5.2. Interrupt Control 5.3. DMA Descriptor Queue 5.4. DMA Control Registers 5.5. Performance Registers
1. Intel® FPGA AI Suite IP Reference Manual
2. About the Intel® FPGA AI Suite IP
3. Intel® FPGA AI Suite IP Generation Utility
4. Intel® FPGA AI Suite Ahead-of-Time Splitter Utility
5. CSR Map and Descriptor Queue
A. Intel® FPGA AI Suite IP Reference Manual Archives
B. Intel® FPGA AI Suite IP Reference Manual Document Revision History

2.5.4.2. Input Folding

Many graphs, particularly those processing image data, have very shallow input channels. In high CVEC instantiations, very shallow input channels can lead to low computational efficiency.

The folding can be done in conjunction with the Intel® FPGA AI Suite compiler and the Intel® FPGA AI Suite IP, or it can be performed by a separate layout transform block that you provide.

Input folding is typically most beneficial to the first layer of a graph.

The following figure illustrates an example of the functionality of the folding transform performed by the Intel® FPGA AI Suite compiler.

Figure 4. Illustration of the folding transform for a 1x1×5×5 input, 1×1×3×3 filter and stride_height = stride_width = 2

In this transformation, the input depth, height, and width are folded into the channel dimension by a factor corresponding to the stride of the first convolution of a network. In the earlier figure, this factor corresponds to transforming the input channels from 1 to 4 (), input height from 5 to 3 () and input width from 5 to 3. Each color corresponds to the new filter window, which in this case would be 4×1×2×2, with the gray boxes corresponding to 0 padding for the filters. Folding is done in a similar way for inputs with depths greater than one, but the folding transform illustration excludes it for simplicity.

The Intel® FPGA AI Suite IP has various enhancements that reduce, but not eliminate the efficiency hit of shallow first layers. In many cases, you can disable first layer folding in the compiler and pass shallow-channel tensors directly to the IP hardware.

You can disable or enable folding with the following Intel® FPGA AI Suite compiler options:

  • NoFolding

    No folding is performed by the host or an external module. This leads to low efficiency for the Intel® FPGA AI Suite IP but might be useful for debugging purposes.

  • ExternalFullFolding

    Folding performed by the host or an external module for the depth, height, and width stride of the first convolution layer.

  • ExternalFullExtraPEFolding

    Folding is performed by the host or an external module for the depth, height, and width stride of the first convolution layer with additional folding performed afterward via the Intel® FPGA AI Suite IP. This might lead to better performance than ExternalFullFolding depending on the instantiation parameters of IP.

  • PEFolding

    Folding is performed entirely by the Intel® FPGA AI Suite IP without the need for any host or external module. This should lead to similar performance to ExternalFullFolding depending on the instantiation parameters of the Intel® FPGA AI Suite IP and the neural network topology.

    PEFolding mode does not support input with a depth greater than one.

Level Two Title