FPGA AI Suite: IP Reference Manual

Download PDF
ID 768974
Date 9/06/2024
Public
Document Table of Contents
Document Table of Contents x
1. FPGA AI Suite IP Reference Manual 2. About the FPGA AI Suite IP 3. FPGA AI Suite IP Generation Utility 4. FPGA AI Suite Ahead-of-Time Splitter Utility 5. CSR Map and Descriptor Queue A. FPGA AI Suite IP Reference Manual Archives B. FPGA AI Suite IP Reference Manual Document Revision History
2. About the FPGA AI Suite IP x
2.1. Supported Models 2.2. Model Performance 2.3. Software Emulation of the FPGA AI Suite IP 2.4. FPGA AI Suite Layer / Primitive Ranges 2.5. FPGA AI Suite IP Block Configuration 2.6. IP Block Interfaces 2.7. Feature Input and Output Streaming 2.8. DDR-Free Operation
2.1. Supported Models x
2.1.1. MobileNet V2 differences between Caffe and TensorFlow models
2.2. Model Performance x
2.2.1. Throughput on the MobileNetV1 model (and other very fast models)
2.5. FPGA AI Suite IP Block Configuration x
2.5.1. Architecture Description File Format for Instance Parameterization 2.5.2. Architecture Description File Parameters
2.5.2. Architecture Description File Parameters x
2.5.2.1. Parameter Group: Global Parameters 2.5.2.2. Parameter Group: activation 2.5.2.3. Parameter Group: pe_array 2.5.2.4. Parameter Group: pool 2.5.2.5. Parameter Group: depthwise 2.5.2.6. Module: softmax 2.5.2.7. Parameter Group: dma 2.5.2.8. Parameter Group: xbar 2.5.2.9. Parameter Group: filter_scratchpad 2.5.2.10. Parameter Group: input_stream_interface 2.5.2.11. Parameter Group: output_stream_interface 2.5.2.12. Parameter Group: config_network 2.5.2.13. (Early Access) Parameter Group: layout_transform_params
2.6. IP Block Interfaces x
2.6.1. Clock and Reset 2.6.2. AXI Interfaces 2.6.3. AXI Interface Clock and Reset 2.6.4. Input Feature Tensor In-Memory Format 2.6.5. Output Tensor In-Memory Format
2.6.4. Input Feature Tensor In-Memory Format x
2.6.4.1. Multiple Input Graphs 2.6.4.2. Input Folding 2.6.4.3. Input Scale and Shift 2.6.4.4. Input Transform Mapping 2.6.4.5. Input Layout Transform Hardware
2.7. Feature Input and Output Streaming x
2.7.1. Input Streaming 2.7.2. Output Streaming
3. FPGA AI Suite IP Generation Utility x
3.1. IP Generation Utility Execution Flows 3.2. IP Generation Utility Inputs 3.3. IP Generation Utility Outputs 3.4. IP Generation Utility Command Line Options
3.4. IP Generation Utility Command Line Options x
3.4.1. The --flow create_ip Flow 3.4.2. The --flow add_arch Flow 3.4.3. The --flow list Flow 3.4.4. The --flow remove_arch Flow
4. FPGA AI Suite Ahead-of-Time Splitter Utility x
4.1. Files Generated by the FPGA AI Suite Ahead-of-Time (AOT) Splitter Utility 4.2. Building the FPGA AI Suite Ahead-of-Time (AOT) Splitter Utility 4.3. Running the FPGA AI Suite Ahead-of-Time (AOT) Splitter Utility 4.4. FPGA AI Suite Ahead-of-Time (AOT) Splitter Utility Example Application
5. CSR Map and Descriptor Queue x
5.1. Discovery ROM 5.2. Interrupt Control 5.3. DMA Descriptor Queue 5.4. DMA Control Registers 5.5. Performance Registers 5.6. Debug Network Registers 5.7. DMA License Register
1. FPGA AI Suite IP Reference Manual
2. About the FPGA AI Suite IP
3. FPGA AI Suite IP Generation Utility
4. FPGA AI Suite Ahead-of-Time Splitter Utility
5. CSR Map and Descriptor Queue
A. FPGA AI Suite IP Reference Manual Archives
B. FPGA AI Suite IP Reference Manual Document Revision History

2.5. FPGA AI Suite IP Block Configuration

The FPGA AI Suite IP block has many important parameters that describe arithmetic precision, feature set, size of various modules (such as the PE Array), and details regarding the internal buses and the external AXI interface.

Configurable parameters are specified in the Architecture Description (.arch) file, as described in Architecture Description File Format for Instance Parameterization and Architecture Description File Parameters.

The table below shows the major parameters, some of which are not configurable, that describe the IP block.

Common Parameter Name

Description

Valid Range

c_vector

(CVEC)

Size of the dot product performed by each PE in the PE Array.

Typically optimized when generating an optimized architecture with the FPGA AI Suite compiler.

[4,8,16,32,64]

k_vector

(PE KVEC)

Number of PEs in the PE Array

[4-128]

Must be a multiple of c_vector

N/A

Number of auxiliary modules connecting to the crossbar (XBAR)

[1-4]

pool k_vector

(Pool KVEC)

Width of the pool interface.

Typically optimized when generating an optimized architecture with the FPGA AI Suite compiler.

[1,2,4,8,16,32,64]

pool max_window_height

pool max_window_width

Size of the pooling window

[3x3, 7x7,13x13]
depthwise k_vector

(Depthwise KVEC)

Number of output channels processed in parallel. Typically optimized when generating an optimized architecture with the FPGA AI Suite compiler. [16, 32, 64]

Must be equal to k_vector

depthwise max_window_height

depthwise max_window_width

Size of the depthwise filter

[3x3, 5x5, 7x7]
depthwise max_dilation_vertical

depthwise max_dilation_horizontal

 Maximum supported value for the depthwise dilation [1-6]

activation k_vector

(Activation KVEC)

Width of the activation interface.

Typically optimized when generating an optimized architecture with the FPGA AI Suite compiler.

[2,4,8,16,32,64]
enable_clamp Enables clamp activation function [true, false]
enable_relu Enables ReLU activation function [true, false]
enable_leaky_relu Enables Leaky ReLU activation function [true, false]
enable_prelu Enables PReLU activation function [true, false]
enable_round_clamp Enables round clamp activation function [true, false]
enable_sigmoid Enables Sigmoid and Swish activation functions [true, false]
enable_tanh Enables Tanh activation function [true, false]
enable_parameter_rom Enables storing graph parameters in on-chip memory, which requires input and output streaming to be enabled and configured.

For details about DDR-free operation, refer to "Generating Artifacts for DDR-Free Operation" in the FPGA AI Suite Compiler Reference Manual .

[true, false]

arch_precision

(PE precision)

Precision of features and weights in the PE Array.

"FP11" (INT7-BFP / 1s.6m.5e)

"FP12AGX" (INT8-BFP / 8m.5e, two's complement)

"FP13AGX" (INT9-BFP / 9m.5e, two's complement)

"FP16" (INT12-BFP / 1s.11m.5e)

PE bias add precision

Precision of accumulator bias value in the PE Array.

fp16

PE accumulator precision

Precision of the accumulators in the PE Array.

fp32

PE drain precision

Precision of values drained from the PE Accumulators to the XBAR and AUX Modules.

fp16
     

PE interleave factor

Multi-threading factor for the features x filters in the PE array accumulators.

Agilex™ 5 devices: 12x1

Agilex™ 7 devices: 2x3, 3x2, 5x1, 1x5

Arria® 10 devices: 2x2, 4x1, 1x4

Stratix® 10 devices: 2x3, 3x2, 5x1, 1x5

1×1 supported for graphs with no bias
PE scale precision Precision of scale multiplier in the PE array fp16

Aux module precision

Precision of the Aux Modules

fp16

Memory port width

Width of memory port

[64, 128, 256, 512]
enable_debug Toggle the FPGA AI Suite debug network that includes interface profiling counters that can be queried with the CSR.

Enabled by default.

 [true, false]
enable_layout_transform Enables the dedicated input tensor layout transform module.
Early access only: This feature has early access support only. Full support for this feature is planned for a future release.
[true, false]

The major constraints include:

  • PE KVEC must be a multiple of CVEC
  • PE KVEC must be divisible by XBAR and AUX KVECs
  • PE drain width must be equal to XBAR KVEC

Graph limitations include:

  • Convolution filter size: 1×1 -> 28×28, including asymmetric
  • Convolution filter stride: 1 .. 15
  • No limitation on convolution padding
  • The limits of the depthwise layers are the same as normal convolution. Depthwise convolution is handled with software emulation using regular convolution passes.

The maximum supported DDR size is 4GB.


Section Content
Architecture Description File Format for Instance Parameterization
Architecture Description File Parameters

Level Two Title