7. Running the Hostless DDR-Free Design Example
The FPGA AI Suite provides a design example to demonstrate hostless and DDR-free operation of the FPGA AI Suite IP. Graph filters, bias, and FPGA AI Suite IP configurations are stored in on-chip memory on the FPGA device instead of DDR memory on the board.
For more details about DDR-free operation, refer to DDR-Free Operation in the FPGA AI Suite IP Reference Manual .
Hardware Requirements
Software Requirements
- FPGA AI Suite
- Quartus® Prime Programmer (either standalone or as part of Quartus® Prime Design Suite).
- Quartus® Prime System Console (either standalone or as part of Quartus® Prime Design Suite).
Procedure
- Generate the parameter ROMs as .mif files by compiling the design example with the following command:
dlac \ --batch-size=1 \ --network-file <path/to/graph> \ --march <path/to/DDR-free/arch> \ --foutput-format=open_vino_hetero \ –o <compiler output .bin file name> \ --fplugin HETERO:FPGA \ --enable-early-access
The .mif files are created in the current directory.
For details about creating the .mif files required for DDR-free operation, refer to "Generating Artifacts for DDR-Free Operation" in the FPGA AI Suite Compiler Reference Manual .
- Build the example design with the following command:
dla_build_example_design.py \ -n 1 \ --arch=<path/to/DDR-free/arch> \ --build --build-dir=<path/to/build/dir> \ --example-design-id=0_STREAMING \ --seed=1 \ --parameter-rom-dir <path/to/mif/files>
Building the example design creates the bitstream needed to program the FPGA device.
For more information about the dla_build_example_design command, refer to "Build Script" in FPGA AI Suite PCIe-based Design Example User Guide .
- Program the FPGA device with the Quartus® Prime Programmer.
The bitstream used to program the device is <path/to/build/dir>/hw/output_files/top.sof.
Program the FPGA device with the following command:quartus_pgm -c 1 -m jtag -o "p;top.sof@1"
For more information about the Quartus® Prime Programmer, refer to Quartus® Prime Pro Edition User Guide: Programmer .
- Use the Quartus® Prime System Console to run inference on the example design.
Because this example design is hostless, operations that typically come from the host are performed through Quartus® Prime System Console instead. For more information about the Quartus® Prime System Console, refer to "Analyzing and Debugging Designs with System Console" in Quartus® Prime Pro Edition User Guide: Debug Tools .
Use the System Console to complete the following steps:- Store input features in the FPGA on-chip memory.
- Prime the FPGA AI Suite IP registers for inference.
- Configure an ingress Modular Scatter-Gather DMA (mSGDMA) core to read the input features from on-chip memory and stream data into the FPGA AI Suite IP.
- Configure an egress mSGDMA core to stream data from the FPGA AI Suite IP into on-chip memory.
- Read the inference results from on-chip memory.
The design example provides a System Console script to automate these operations for you. You can find the script in the $CORDLA_ROOT/runtime/stream/ed0_streaming_example folder.
If you want to use the provided script for you own design, ensure that you review the information about the mSGDMA core, refer to "Modular Scatter-Gather DMA Core" in Embedded Peripherals IP User Guide to understand how to extend the script for your design.
To use the design example System Console script:- Copy a raw img.bin file to the $COREDLA_ROOT/runtime/streaming/ed0_streaming_example directory. The System Console script uses the img.bin file as input to the model.
- Run the following command:
system-console –-script=system_console_script.tcl
The design example System Console script generates a file called output.bin that contains the raw inference results.
- (Optional) Postprocess the raw inference output for readability.
The design example provides the $COREDLA_ROOT/bin/streaming_post_processing.py script to clean the raw output.bin file. This script removes some invalid bytes and stores an FP16 formatted result_hw.txt file for readability.