FPGA AI Suite: PCIe-based Design Example User Guide

Download PDF
ID 768977
Date 7/31/2024
Public
Document Table of Contents
Document Table of Contents x
1. FPGA AI Suite PCIe-based Design Example User Guide 2. About the PCIe* -based Design Example 3. Getting Started with the FPGA AI Suite PCIe* -based Design Example 4. Building the FPGA AI Suite Runtime 5. Running the Design Example Demonstration Applications 6. Design Example Components 7. Design Example System Architecture for the Agilex™ 7 FPGA A. FPGA AI Suite PCIe-based Design Example User Guide Archives B. FPGA AI Suite PCIe-based Design Example User Guide Document Revision History
4. Building the FPGA AI Suite Runtime x
4.1. CMake Targets 4.2. Build Options
5. Running the Design Example Demonstration Applications x
5.1. Exporting Trained Graphs from Source Frameworks 5.2. Compiling Exported Graphs Through the FPGA AI Suite 5.3. Compiling the PCIe* -based Example Design 5.4. Programming the FPGA Device ( Agilex™ 7) 5.5. Performing Accelerated Inference with the dla_benchmark Application 5.6. Running the Ported OpenVINO™ Demonstration Applications
5.5. Performing Accelerated Inference with the dla_benchmark Application x
5.5.1. Inference on Image Classification Graphs 5.5.2. Inference on Object Detection Graphs 5.5.3. Additional dla_benchmark Options 5.5.4. The dla_benchmark Performance Metrics
5.5.2. Inference on Object Detection Graphs x
5.5.2.1. The mAP and COCO AP Metrics 5.5.2.2. Specifying Ground Truth 5.5.2.3. Example of Inference on Object Detection Graphs
5.5.4. The dla_benchmark Performance Metrics x
5.5.4.1. Interpreting System Throughput and Latency Metrics
5.6. Running the Ported OpenVINO™ Demonstration Applications x
5.6.1. Example Running the Object Detection Demonstration Application
6. Design Example Components x
6.1. Build Script 6.2. Example Architecture Bitstream Files 6.3. Software Components 6.4. Software Interface to the BSP
6.1. Build Script x
6.1.1. Build Script Options 6.1.2. Script Flow
6.3. Software Components x
6.3.1. OpenVINO™ FPGA Runtime Plugin 6.3.2. FPGA AI Suite Runtime 6.3.3. BSP Driver
7. Design Example System Architecture for the Agilex™ 7 FPGA x
7.1. System Overview 7.2. Hardware
7.2. Hardware x
7.2.1. PLL Adjustment
1. FPGA AI Suite PCIe-based Design Example User Guide
2. About the PCIe* -based Design Example
3. Getting Started with the FPGA AI Suite PCIe* -based Design Example
4. Building the FPGA AI Suite Runtime
5. Running the Design Example Demonstration Applications
6. Design Example Components
7. Design Example System Architecture for the Agilex™ 7 FPGA
A. FPGA AI Suite PCIe-based Design Example User Guide Archives
B. FPGA AI Suite PCIe-based Design Example User Guide Document Revision History

5.5.4. The dla_benchmark Performance Metrics

The -save_run_summary option makes the dla_benchmark demonstration application collect performance metrics during inference. These metrics can help you determine how efficient an architecture is at executing a model.

Note: The dla_benchmark application provides throughput in "frames per second". The time per frame (latency) is 1/throughput.

Statistic

Description

Count

The number of times interference was performed. This is set by the -niter option.

System duration

The total time between when the first inference request was made to when the last request was finished, as measured by the host program.

IP duration

The total time the spent-on inference. This is reported by the IP on the FPGA.

Latency

The median time of all inference requests made by the host. This includes any overhead from OpenVINO™ or the FPGA AI Suite runtime.

System throughput

The total throughput of the system, including any OpenVINO™ or FPGA AI Suite runtime overhead.

Number of hardware instances

The number of IP instances on the FPGA.

Number of network instances

The number graphs that the IP processes in parallel.

IP throughput per instance

The throughput of a single IP instance. This is reported by the IP on the FPGA.

IP throughput per fMAX per instance

The IP throughput per instance value scaled by the IP clock frequency value.

IP clock frequency

The clock frequency, as reported by the IP running on the FPGA device.

The dla_benchmark application treats this value as the IP core fMAX value.

Estimated IP throughput per instance

The estimated per-IP throughput, as estimated by the dla_compiler command with the --fanalyze-performance option.

Estimated IP throughput per fmax per instance

The Estimated IP throughput per instance value scaled by the compiler fMAX estimate.


Section Content
Interpreting System Throughput and Latency Metrics

Level Two Title