7. Running the Hostless Memoryless Design Example
The FPGA AI Suite provides a design example to demonstrate hostless and memoryless operation of the FPGA AI Suite IP. Graph filters, bias, and FPGA AI Suite IP configurations are stored in on-chip memory on the FPGA device instead of DDR memory on the board.
For more details about memoryless operation, refer to Memoryless Operation in the FPGA AI Suite IP Reference Manual .
Hardware Requirements
Software Requirements
- FPGA AI Suite
- Quartus® Prime Programmer (either standalone or as part of Quartus® Prime Design Suite).
- Quartus® Prime System Console (either standalone or as part of Quartus® Prime Design Suite).
Procedure
- Generate the parameter ROMs as .mif files by compiling the design example with the following command:
dlac \ --batch-size=1 \ --network-file <path/to/graph> \ --march <path/to/memoryless/arch> \ --foutput-format=open_vino_hetero \ –o <compiler output .bin file name> \ --fplugin HETERO:FPGA \ --enable-early-access
The .mif files are created in the current directory.
For details about creating the .mif files required for memoryless operation, refer to "Generating Artifacts for Memoryless Operation" in the FPGA AI Suite Compiler Reference Manual .
- Build the example design with the following command:
dla_build_example_design.py \ -n 1 \ --arch=<path/to/memoryless/arch> \ --build --build-dir=<path/to/build/dir> \ --example-design-id=0_STREAMING \ --seed=1 \ --parameter-rom-dir <path/to/mif/files>
Building the example design creates the bitstream needed to program the FPGA device.
For more information about the dla_build_example_design command, refer to "Build Script" in FPGA AI Suite PCIe-based Design Example User Guide .
- Program the FPGA device with the Quartus® Prime Programmer.
The bitstream used to program the device is <path/to/build/dir>/hw/output_files/top.sof.
Program the FPGA device with the following command:quartus_pgm -c 1 -m jtag -o "p;top.sof@1"
For more information about the Quartus® Prime Programmer, refer to Quartus® Prime Pro Edition User Guide: Programmer .
- Use the Quartus® Prime System Console to run inference on the example design.
Because this example design is hostless, operations that typically come from the host are performed through Quartus® Prime System Console instead. For more information about the Quartus® Prime System Console, refer to "Analyzing and Debugging Designs with System Console" in Quartus® Prime Pro Edition User Guide: Debug Tools .
Use the System Console to complete the following steps:- Store input features in the FPGA on-chip memory.
- Prime the FPGA AI Suite IP registers for inference.
- Configure an ingress Modular Scatter-Gather DMA (mSGDMA) core to read the input features from on-chip memory and stream data into the FPGA AI Suite IP.
- Configure an egress mSGDMA core to stream data from the FPGA AI Suite IP into on-chip memory.
- Read the inference results from on-chip memory.
The design example provides a System Console script to automate these operations for you. You can find the script in the $CORDLA_ROOT/runtime/stream/ed0_streaming_example folder.
If you want to use the provided script for you own design, ensure that you review the information about the mSGDMA core, refer to "Modular Scatter-Gather DMA Core" in Embedded Peripherals IP User Guide to understand how to extend the script for your design.
To use the design example System Console script:- Copy a raw img.bin file to the $COREDLA_ROOT/runtime/streaming/ed0_streaming_example directory. The System Console script uses the img.bin file as input to the model.
- Run the following command:
system-console –-script=system_console_script.tcl
The design example System Console script generates a file called output.bin that contains the raw inference results.
- (Optional) Postprocess the raw inference output for readability.
The design example provides the $COREDLA_ROOT/bin/streaming_post_processing.py script to clean the raw output.bin file. This script removes some invalid bytes and stores an FP16 formatted result_hw.txt file for readability.