FPGA AI Suite: IP Reference Manual

ID 768974
Date 9/06/2024
Public
Document Table of Contents

2.8. DDR-Free Operation

To avoid use of external memory for storing graph weights and FPGA AI Suite IP configurations during inference, you can store these parameters within the IP using on-chip memory.

In this case, three types of memory initialization files (.mif) are required:

  • memoryless_filter_hw*.mif: Contains the graph filters.
  • memoryless_bias_scale_hw*.mif: Contains the graph biases and scaling factors.
  • memoryless_config.mif: Stores the FPGA AI Suite instructions for the compiled graph.

These files are generated using the dla_compiler tool that takes the architecture definition (.arch file) and the target neural network graph as inputs.

A total of K_VECTOR DDR-free filter memory initialization files (.mif) files are generated. Each file is postfixed with an integer representing which PE the filter is to be loaded to.

Similarly, a total of K_VECTOR DDR-free bias and scale files are generated, using the same postfix scheme to indicate the PE that should load this file.

The FPGA AI Suite IP requires a stream of instructions that describe the order in which convolutions, activations, and other operations must be performed. This instruction stream is stored in a single .mif file (memoryless_config.mif).

These files are used to initialize read-only memories while building an FPGA bitstream, which means that bitstreams are graph-specific for DDR-free operation.

You must also enable input and output streaming when using DDR-free operation. The graph must have a large enough stream buffer depth to accommodate all of the intermediate results during inference.

The dla_build_example_design.py command can build design examples with DDR-free operation. The only streaming design example option currently supported is the 0_STREAMING option. The design example created by this option targets an Agilex™ 7 FPGA I-Series Development Kit (DK-DEV-AGI027RBES).

To compile a bitstream with a DDR-free architecture, specify the directory that contains the DDR-free .mif files with the –parameter_rom_dir option of the dla_build_example_design.py command.

For details about creating the .mif file required for DDR-free operation, refer to "Generating Artifacts for DDR-Free Operation" in the FPGA AI Suite Compiler Reference Manual .