FPGA AI Suite: Getting Started Guide

ID 768970
Date 9/06/2024
Public
Document Table of Contents

6.13. Performing Inference Without an FPGA Board

The software emulation model is a C++ software model of the FPGA AI Suite IP that is bit-accurate*. The emulation models relatively low-level transactions and, in some cases, models processing delays of modules.

To use the software emulation model:
  1. Build the runtime with the following commands:
    cd $COREDLA_WORK/runtime
    rm -rf build_Release
    ./build_runtime.sh -target_reference
  2. Run inference with the -niter=1 and -nireq=1 options (because the emulation is reasonably slow) with the following commands:
    modeldir=$COREDLA_WORK/demo/models/public
    imagedir=$COREDLA_WORK/demo/sample_images
    curarch=$COREDLA_ROOT/example_architectures/AGX7_Performance.arch
    gnd=$imagedir/TF_ground_truth.txt
    
    ./dla_benchmark \
       -b 1  `# Run only a single image` \
       -niter 1  `# Run only a single image` \
       -nireq 1  `# Running reference: so -nireq=1` \
       -m $modeldir/resnet-50-tf/FP32/resnet-50-tf.xml \
        `# Same as when running on hardware - specify the graph` \
       -d HETERO:FPGA,CPU \
        `# Same as when running on hardware - \
        use FPGA if possible, fallback to CPU` \
       -i $imagedir  `# Same as when running on hardware - specify image directory` \
       -arch_file $curarch  `# Same as when running on hardware - specify .arch file` \
       -dump_output  `# Dump output result.txt file` \
       -plugins reference  `# Use the software reference` \
       -groundtruth_loc $gnd  `# Location of the ground truth file for $imagedir`
* Minor rounding differences between software emulation and hardware will typically result in differences of less than two units of least precision (ulps.