Multi Channel DMA for PCI Express* Intel® FPGA IP Design Example User Guide

Download PDF
ID 683517
Date 12/01/2021
Public

A newer version of this document is available. Customers should click here to go to the newest version.

Document Table of Contents
Document Table of Contents x
1. Terms and Acronyms 2. Design Example Detailed Description 3. Design Example Quick Start Guide 4. Multi Channel DMA for FPGA IP Design Example User Guide Archives 5. Document Revision History for the Multi Channel DMA for FPGA IP Design Example User Guide
2. Design Example Detailed Description x
2.1. Design Example Overview 2.2. Hardware and Software Requirements 2.3. PIO Using MCDMA Bypass Mode 2.4. Avalon-ST Packet Generate/Check 2.5. Avalon-ST Device-side Packet Loopback 2.6. Avalon-MM DMA 2.7. Traffic Generator and Checker
2.3. PIO Using MCDMA Bypass Mode x
2.3.1. Avalon-ST PIO Using MCDMA Bypass Mode 2.3.2. Avalon-MM PIO Using MCDMA Bypass mode
2.3.1. Avalon-ST PIO Using MCDMA Bypass Mode x
2.3.1.1. Four-Port Avalon-ST PIO Using MCDMA Bypass Mode 2.3.1.2. Single-Port Avalon-ST PIO Using MCDMA Bypass Mode
2.3.1.1. Four-Port Avalon-ST PIO Using MCDMA Bypass Mode x
2.3.1.1.1. Simulation Result 2.3.1.1.2. Hardware Test Result
2.3.2. Avalon-MM PIO Using MCDMA Bypass mode x
2.3.2.1. Simulation Results 2.3.2.2. Hardware Test Results
2.4. Avalon-ST Packet Generate/Check x
2.4.1. Four-Port Avalon-ST Packet Generate/Check 2.4.2. Single-Port Avalon-ST Packet Generate/Check
2.4.1. Four-Port Avalon-ST Packet Generate/Check x
2.4.1.1. Simulation Results 2.4.1.2. Hardware Test Results
2.4.2. Single-Port Avalon-ST Packet Generate/Check x
2.4.2.1. Simulation Waveforms 2.4.2.2. Simulation Log
2.5. Avalon-ST Device-side Packet Loopback x
2.5.1. Simulation Results 2.5.2. Hardware Test Results
2.6. Avalon-MM DMA x
2.6.1. Simulation Results 2.6.2. Hardware Test Results
2.7. Traffic Generator and Checker x
2.7.1. Traffic Generator and checker Example Design Register Map
3. Design Example Quick Start Guide x
3.1. Design Example Directory Structure 3.2. Generating the Example Design using Intel® Quartus® Prime 3.3. Simulating the Design Example 3.4. Compiling the Example Design in Intel® Quartus® Prime 3.5. Running the Design Example Application on a Hardware Setup
3.2. Generating the Example Design using Intel® Quartus® Prime x
3.2.1. Procedure
3.3. Simulating the Design Example x
3.3.1. Testbench Overview 3.3.2. Supported Simulators 3.3.3. Example Testbench Flow for DMA Test with Avalon-ST Packet Generate/Check Design Example 3.3.4. Run the Simulation Script 3.3.5. View the Results
3.5. Running the Design Example Application on a Hardware Setup x
3.5.1. Program the FPGA 3.5.2. Quick Start Guide
3.5.2. Quick Start Guide x
3.5.2.1. Software Test Setup 3.5.2.2. Driver Support 3.5.2.3. MCDMA Custom Driver 3.5.2.4. MCDMA DPDK Poll Mode Driver 3.5.2.5. MCDMA Kernel Mode Character Device Driver 3.5.2.6. MCDMA Kernel Mode Network Device Driver
3.5.2.3. MCDMA Custom Driver x
3.5.2.3.1. Prerequisites 3.5.2.3.2. Software Setup 3.5.2.3.3. Run the Reference Example Application 3.5.2.3.4. 2K Channels Test with DIDF Mode
3.5.2.3.1. Prerequisites x
3.5.2.3.1.1. Configuration Changes from BIOS 3.5.2.3.1.2. External Packages 3.5.2.3.1.3. Set the Boot Parameters
3.5.2.3.2. Software Setup x
3.5.2.3.2.1. Installing the Linux Kernel Driver and Enabling VFs 3.5.2.3.2.2. Build and Install User Space Library 3.5.2.3.2.3. Create Guest VM by Using QEMU 3.5.2.3.2.4. Establish Communication Between Host and QEMU
3.5.2.3.3. Run the Reference Example Application x
3.5.2.3.3.1. Meta Data Test 3.5.2.3.3.2. BAM Test 3.5.2.3.3.3. BAS Test 3.5.2.3.3.4. Packet Gen Test
3.5.2.4. MCDMA DPDK Poll Mode Driver x
3.5.2.4.1. Prerequisites 3.5.2.4.2. Install PMD and Test Application 3.5.2.4.3. Run the Reference Example Application
3.5.2.4.1. Prerequisites x
3.5.2.4.1.1. Set the Boot Parameters 3.5.2.4.1.2. Configuration Changes from BIOS 3.5.2.4.1.3. Install and Build Testpmd
3.5.2.4.2. Install PMD and Test Application x
3.5.2.4.2.1. Create VM Environment
3.5.2.4.3. Run the Reference Example Application x
3.5.2.4.3.1. Example of Verifying on an AVMM Design 3.5.2.4.3.2. BAM Test 3.5.2.4.3.3. BAS Test
3.5.2.5. MCDMA Kernel Mode Character Device Driver x
3.5.2.5.1. Prerequisites 3.5.2.5.2. Software Setup 3.5.2.5.3. Examples
3.5.2.5.1. Prerequisites x
3.5.2.5.1.1. Host Configuration 3.5.2.5.1.2. Set the Boot Parameters
3.5.2.5.2. Software Setup x
3.5.2.5.2.1. Build and Install Kernel Driver 3.5.2.5.2.2. Build and Install Libmcmem 3.5.2.5.2.3. Build Testapp 3.5.2.5.2.4. Configure Kernel Driver 3.5.2.5.2.5. Configure Libmcmem 3.5.2.5.2.6. Configure Testapp
3.5.2.6. MCDMA Kernel Mode Network Device Driver x
3.5.2.6.1. Build and Install Netdev Driver 3.5.2.6.2. Configure the Device Communication 3.5.2.6.3. Test Procedure by Using Name Space Environment
1. Terms and Acronyms
2. Design Example Detailed Description
3. Design Example Quick Start Guide
4. Multi Channel DMA for FPGA IP Design Example User Guide Archives
5. Document Revision History for the Multi Channel DMA for FPGA IP Design Example User Guide

2.1. Design Example Overview

The Multi Channel DMA for PCI Express IP Design Examples demonstrate a Multi Channel DMA solution for Intel® Stratix® 10 GX/MX devices using the H-Tile PCIe Gen3 hard IP, Intel® Stratix® 10 DX devices using the P-Tile PCIe Gen4 hard IP, and Intel® Agilex™ devices using the P-Tile or F-Tile PCIe Gen4 Hard IP and soft IP implemented in the FPGA fabric.

You can generate the design example from the Example Designs tab of the Multi Channel DMA for PCI Express IP Parameter Editor. The desired user interface type, either Avalon-ST or Avalon-MM, can be chosen. You can allocate up to 2048 DMA channels (with a maximum of 512 channels per function) when the Avalon-MM interface type or Avalon-ST 1-port interface is selected. For the Avalon-ST 4-port Interface, one channel is allocated per port.

Table 2.  Multi Channel DMA for PCI Express* IP Design Example Variants
Design Example Configuration Simulation Driver Support
H-Tile P-Tile F-Tile
AVMM DMA

User Mode: Multi channel DMA

Interface Type: Avalon-MM

Number of Ports: 1

SR-IOV Support: No

Total Channels Supported: up to 512 channels

 Yes up to 512 channels Simulator: VCS, Questa*-Intel® FPGA Starter EditionQuestaSim* (H-Tile only), Xcelium* (H-Tile only)
  • Custom
  • DPDK
  • Kernel Mode

User Mode: Multi channel DMA

Interface Type: Avalon-MM

Number of Ports: 1

SR-IOV Support: Yes 1

Total Channels Supported: up to 2K channels

 One Physical Function and its Virtual Functions 2 Simulator: VCS, QuestaSim* (H-Tile only), Xcelium (H-Tile only) Not Supported
  • Custom
  • DPDK

User Mode: BAM+ MCDMA

Interface Type: Avalon-MM

Number of Ports: 1

SR-IOV Support: No

Total Channels Supported: up to 512 channels

 Yes up to 512 channels Simulator: VCS, QuestaSim* (H-Tile only), Xcelium (H-Tile only)
  • Custom
  • DPDK
  • Kernel Mode

User Mode: BAM+ MCDMA

Interface Type: Avalon-MM

Number of Ports: 1

SR-IOV Support: Yes1

Total Channels Supported: up to 2K channels2

One Physical Function and its Virtual Functions Simulator: VCS, QuestaSim* (H-Tile only), Xcelium (H-Tile only) Not Supported
  • Custom
  • DPDK
Device-side Packet loopback

User Mode: Multi channel DMA

Interface Type: Avalon-ST

Number of Ports: 4

SR-IOV Support: No

Total Channels Supported: 1 Channel per port

 One channel per port Simulator: VCS, QuestaSim* (H-Tile only), Xcelium (H-Tile only)
  • Custom
  • DPDK
  • Kernel Mode

User Mode: Multi channel DMA

Interface Type: Avalon-ST

Number of Ports: 1

SR-IOV Support: No

Total Channels Supported: up to 512 channels2

Yes up to 512 channels Simulator: VCS, QuestaSim* (H-Tile only), Xcelium (H-Tile only)
  • Custom
  • DPDK
  • Kernel Mode
  • Netdev (Support four PFs, one channel per PF)

User Mode: Multi channel DMA

Interface Type: Avalon-ST

Number of Ports: 1

SR-IOV Support: Yes

Total Channels Supported: up to 2K channels2

One Physical Function and its Virtual Functions Simulator: VCS, QuestaSim* (H-Tile only), Xcelium (H-Tile only) Not Supported
  • Custom
  • DPDK
Packet Generate/Check

User Mode: Multi channel DMA

Interface Type: Avalon-ST

Number of Ports: 4

SR-IOV Support: No

Total Channels Supported: 1 channel per port

 One channel per port Simulator: VCS, QuestaSim* (H-Tile only), Xcelium (H-Tile only)
  • Custom
  • DPDK
  • Kernel Mode

User Mode: Multi channel DMA

Interface Type: Avalon-ST

Number of Ports: 1

SR-IOV Support: No

Total Channels Supported: up to 512 channels

 Yes up to 512 channels Simulator: VCS, QuestaSim* (H-Tile only), Xcelium (H-Tile only)
  • Custom
  • DPDK
  • Kernel Mode

User Mode: Multi channel DMA

Interface Type: Avalon-ST

Number of Ports: 1

SR-IOV Support: Yes1

Total Channels Supported: up to 2K channels2

One Physical Function and its Virtual Functions Simulator: VCS, QuestaSim* (H-Tile only), Xcelium (H-Tile only) Not Supported
  • Custom
  • DPDK
PIO using MQDMA Bypass mode

User Mode: Multi channel DMA

Interface Type: Avalon-ST

Number of Ports: 4

SR-IOV Support: No

 Yes Simulator: VCS, QuestaSim*, Xcelium (H-Tile only)
  • Custom
  • DPDK
  • Kernel Mode

User Mode: Multi channel DMA

Interface Type: Avalon-ST

Number of Ports: 1

SR-IOV Support: No

 Yes Simulator: VCS, QuestaSim*, Xcelium (H-Tile only)
  • Custom
  • DPDK
  • Kernel Mode

User Mode: BAM+ MCDMA

Interface Type: Avalon-ST

Number of Ports: 1

SR-IOV Support: No

 Yes Simulator: VCS, QuestaSim*, Xcelium (H-Tile only)
  • Custom
  • DPDK
  • Kernel Mode

User Mode: BAM+ MCDMA

Interface Type: Avalon-ST

Number of Ports: 4

SR-IOV Support: No

 Yes Simulator: VCS, QuestaSim*, Xcelium (H-Tile only)
  • Custom
  • DPDK
  • Kernel Mode

User Mode: Bursting Master

Interface Type: Avalon-MM

Number of Ports: N/A

SR-IOV Support: No

 Yes Simulator: VCS, QuestaSim*, Xcelium (H-Tile only)
  • Custom
  • DPDK
  • Kernel Mode

User Mode: BAM+BAS

Interface Type: Avalon-MM

Number of Ports: N/A

 Yes Simulator: VCS, QuestaSim*, Xcelium (H-Tile only)
  • Custom
  • DPDK
  • Kernel Mode

User Mode: BAM+ MCDMA

Interface Type: Avalon-MM

Number of Ports: N/A

 Yes Simulator: VCS, QuestaSim*, Xcelium (H-Tile only)
  • Custom
  • DPDK
  • Kernel Mode
Traffic Generator/Checker

User Mode: BAM+BAS Interface

Type: Avalon-MM

Number of Ports: N/A

SR-IOV Support: No

 Yes Simulator: VCS, Xcelium (H-Tile only)
  • Custom
  • DPDK
  • Kernel Mode

User Mode: BAM+BAS Interface

Type: Avalon-MM

Number of Ports: N/A

SR-IOV Support: Yes1

Not Supported3
  • Custom
  • DPDK
1 Refer to P-tile Avalon® Streaming Intel® FPGA IP for PCI Express* User Guide for supported physical function or virtual function combinations.
2 512 channels per function
3 For more detailed simulators support availability, please refer to and
Level Two Title