Multi Channel DMA Intel® FPGA IP for PCI Express* Design Example User Guide

Download PDF
ID 683517
Date 7/30/2024
Public
Document Table of Contents
Document Table of Contents x
1. Introduction 2. Design Example Detailed Description 3. Design Example Quick Start Guide 4. Multi Channel DMA Intel FPGA IP for PCI Express Design Example User Guide Archives 5. Revision History for the Multi Channel DMA Intel FPGA IP for PCI Express Design Example User Guide
1. Introduction x
1.1. Terms and Acronyms 1.2. MCDMA IP Modes
2. Design Example Detailed Description x
2.1. Design Example Overview 2.2. Hardware and Software Requirements 2.3. PIO using MCDMA Bypass Mode 2.4. Avalon-ST Packet Generate/Check 2.5. Avalon-ST Device-side Packet Loopback 2.6. Avalon-MM DMA 2.7. BAM_BAS Traffic Generator and Checker 2.8. External Descriptor Controller
2.1. Design Example Overview x
2.1.1. H-Tile MCDMA IP - Design Examples for Endpoint 2.1.2. P-Tile MCDMA IP - Design Examples for Endpoint 2.1.3. F-Tile MCDMA IP - Design Examples for Endpoint 2.1.4. R-Tile MCDMA IP - Design Examples for Endpoint 2.1.5. Design Example BAR Mappings
2.3. PIO using MCDMA Bypass Mode x
2.3.1. Avalon-ST PIO using MCDMA Bypass Mode 2.3.2. Avalon-MM PIO using MCDMA Bypass Mode
2.3.1. Avalon-ST PIO using MCDMA Bypass Mode x
2.3.1.1. Single-Port Avalon-ST PIO using MCDMA Bypass Mode
2.3.2. Avalon-MM PIO using MCDMA Bypass Mode x
2.3.2.1. Simulation Results 2.3.2.2. Hardware Test Results
2.4. Avalon-ST Packet Generate/Check x
2.4.1. Single-Port Avalon-ST Packet Generate/Check
2.4.1. Single-Port Avalon-ST Packet Generate/Check x
2.4.1.1. Simulation Waveforms 2.4.1.2. Simulation Log 2.4.1.3. Hardware Test Result
2.5. Avalon-ST Device-side Packet Loopback x
2.5.1. Simulation Results 2.5.2. Hardware Test Results
2.6. Avalon-MM DMA x
2.6.1. Simulation Results 2.6.2. Hardware Test Results
2.7. BAM_BAS Traffic Generator and Checker x
2.7.1. BAM_BAS Traffic Generator and Checker Example Design Register Map
2.8. External Descriptor Controller x
2.8.1. Registers 2.8.2. Hardware Test Results
2.8.1. Registers x
2.8.1.1. GCSR Registers (Base address 64’h00000) 2.8.1.2. QCSR Registers(Base address 64’h10000)
3. Design Example Quick Start Guide x
3.1. Design Example Directory Structure 3.2. Generating the Example Design using Quartus® Prime 3.3. Simulating the Design Example 3.4. Compiling the Example Design in Quartus® Prime 3.5. Running the Design Example Application on a Hardware Setup
3.2. Generating the Example Design using Quartus® Prime x
3.2.1. Procedure
3.3. Simulating the Design Example x
3.3.1. Testbench Overview 3.3.2. Supported Simulators 3.3.3. Example Testbench Flow for DMA Test with Avalon-ST Packet Generate/Check Design Example 3.3.4. Run the Simulation Script 3.3.5. Steps to Run the Simulation 3.3.6. View the Results
3.3.5. Steps to Run the Simulation x
3.3.5.1. Steps to Run the Simulation : QuestaSim* / Questa* Intel® FPGA Edition 3.3.5.2. Steps to Run the Simulation : VCS* / VCS* MX 3.3.5.3. Steps to Run the Simulation : Xcelium*
3.5. Running the Design Example Application on a Hardware Setup x
3.5.1. Program the FPGA 3.5.2. Quick Start Guide
3.5.2. Quick Start Guide x
3.5.2.1. Software Test Setup 3.5.2.2. Driver Support 3.5.2.3. MCDMA Custom Driver 3.5.2.4. MCDMA DPDK Poll Mode Driver 3.5.2.5. MCDMA Kernel Mode Network Device Driver
3.5.2.3. MCDMA Custom Driver x
3.5.2.3.1. Prerequisites 3.5.2.3.2. Software Setup 3.5.2.3.3. Run the Reference Example Application 3.5.2.3.4. Testing Bitstream Configuration beyond 256 Channels (for MCDMA Custom Driver) 3.5.2.3.5. External Descriptor Mode Verification
3.5.2.3.1. Prerequisites x
3.5.2.3.1.1. Configuration Changes from BIOS 3.5.2.3.1.2. External Packages 3.5.2.3.1.3. Set the Boot Parameters
3.5.2.3.2. Software Setup x
3.5.2.3.2.1. Installing the Linux Kernel Driver 3.5.2.3.2.2. Build and Install User Space Library 3.5.2.3.2.3. Enabling VFs and Create Guest VM by Using QEMU 3.5.2.3.2.4. Establish Communication Between Host and QEMU
3.5.2.3.3. Run the Reference Example Application x
3.5.2.3.3.1. Meta Data Test 3.5.2.3.3.2. Custom PIO Read Write Test 3.5.2.3.3.3. BAM Test 3.5.2.3.3.4. BAS Test 3.5.2.3.3.5. Packet Gen Test
3.5.2.4. MCDMA DPDK Poll Mode Driver x
3.5.2.4.1. Prerequisites 3.5.2.4.2. Install PMD and Test Application (for CentOS) 3.5.2.4.3. Install PMD and Test Application (for Ubuntu) 3.5.2.4.4. Create VM Environment 3.5.2.4.5. Run the Reference Example Application
3.5.2.4.1. Prerequisites x
3.5.2.4.1.1. Set the Boot Parameters 3.5.2.4.1.2. Configuration Changes from BIOS 3.5.2.4.1.3. Install and Build Testpmd
3.5.2.4.4. Create VM Environment x
3.5.2.4.4.1. Enable VFs 3.5.2.4.4.2. Create Guest VM by using QEMU 3.5.2.4.4.3. Establish Communication Between Host and VM
3.5.2.4.5. Run the Reference Example Application x
3.5.2.4.5.1. Channel ID VF PF Verification 3.5.2.4.5.2. Example of Verifying on an AVMM Design 3.5.2.4.5.3. Testing Bitstream Configuration beyond 256 Channels (for DPDK Poll Mode Driver) 3.5.2.4.5.4. BAM Test 3.5.2.4.5.5. BAS Test
3.5.2.5. MCDMA Kernel Mode Network Device Driver x
3.5.2.5.1. Build and Install Netdev Driver 3.5.2.5.2. Enable VFs if SRIOV is Supported 3.5.2.5.3. Configure the Number of Channels Supported on the Device 3.5.2.5.4. Configure the MTU Value 3.5.2.5.5. Configure the Device Communication 3.5.2.5.6. Configure Transmit Queue Selection Mechanism 3.5.2.5.7. Test Procedure by Using Name Space Environment 3.5.2.5.8. PIO Test
3.5.2.5.6. Configure Transmit Queue Selection Mechanism x
3.5.2.5.6.1. MCDMA Queue Selection 3.5.2.5.6.2. Transmit Packet Steering (XPS) 3.5.2.5.6.3. Default
1. Introduction
2. Design Example Detailed Description
3. Design Example Quick Start Guide
4. Multi Channel DMA Intel FPGA IP for PCI Express Design Example User Guide Archives
5. Revision History for the Multi Channel DMA Intel FPGA IP for PCI Express Design Example User Guide

3.5.2.2. Driver Support

Check the kernel version using this command:

$ uname -r

Expected output:

5.15.0-xx-generic

If this is not the kernel version in your Ubuntu 22.04 system, follow the steps below. These steps will change the kernel from HWE to GA Linux 5.15.0-xx-generic and install Linux headers and gcc required for the MCDMA drivers.

sudo apt install linux-image-generic

// Install GA.

// Reboot into the newly-installed older kernel.

sudo apt remove linux-generic-hwe-22.04

// Remove HWE.

sudo apt autoremove

// Clean up.

sudo apt-get install linux-headers-generic gcc

Check the gcc version using this command:

$ gcc --version

If the gcc is not gcc-11, install gcc-11 using this command:

$ sudo apt install gcc-11

To switch between the installed gcc versions, use the update-alternatives tool and select gcc-11.

$ sudo update-alternatives --config gcc

Note: Ensure that Proxy is set. Otherwise, some of these updates do not work.
The table below summarizes the driver support for the MCDMA design example variants. It uses the following acronyms:
  • User Space I/O (UIO): A kernel base module that the PCIe device uses to expose its resources to user space.
  • Virtual Function I/O (VFIO) driver: An IOMMU/device agnostic framework for exposing direct device access to user space in a secure, IOMMU-protected environment.
  • Data Plane Development Kit (DPDK): Consists of libraries to accelerate packet processing workloads running on a wide variety of CPU architectures.
Note:
Software directory is created multiple times depending on Hard IP mode selected (1x16, 2x8 or 4x4) for Quartus® Prime Pro Edition 23.3 version onwards.
  • p0_software folder is generated for 1x16 Hard IP modes.
  • p1_software folder is generated for 2x8 Hard IP modes.
  • p2_software and p3_software folders are generated only for 4x4 Hard IP modes.
Note: You must use the corresponding software folder with each IP port.
Table 41.  Driver Support for MCDMA Design Examples
Parameter Custom Driver DPDK Driver Kernel Mode Netdev Driver
Description Also known as the user mode driver, this driver is created to support both UIO and VFIO base kernelmodules. This driver provides custom APIs and can be used without depending on any framework. This DPDK Poll Mode Driver (PMD) uses the DPDK framework. The PMD will expose the device as an ethernet device. It supports both UIO and VFIO base kernel modules. Existing DPDK applications can be integrated with the MCDMA PMD. Kernel Mode Netdev Driver exposes the MCDMA IP as a Network Device and enables standard applications to perform network data transfers using the Linux network stack.
Directory/Driver Path <example_design>/pX_software/user <example_design>/pX_software/dpdk <example design>/pX_software/kernel/
SR-IOV Support Yes Yes Yes
Multi channel DMA Avalon-MM DMA Design Example Yes, up to 2K Channels Yes, up to 2K Channels No
Multi channel DMA Avalon-MM DMA with SRIOV Design Example Yes, up to 2K Channels Yes, up to 2K Channels No
BAM+BAS+MCDMA Avalon-MM DMA Design Example Yes, up to 2K Channels Yes, up to 2K Channels No
BAM+BAS+MCDMA Avalon-MM DMA with SR-IOV Design Example Yes, up to 2K Channels Yes, up to 2K Channels No
Multi channel DMA Avalon-MM PIO using MQDMA Bypass Mode Design Example Yes Yes No
Multi channel DMA Avalon-ST 1-port PIO using MQDMA Bypass Mode Design Example Yes Yes Yes
BAM+BAS+MCDMA Avalon-MM PIO using MQDMA Bypass Mode Design Example Yes Yes No
BAM+BAS+MCDMA Avalon-ST 1-Port PIO using MQDMA Bypass Mode Design Example Yes Yes No
Bursting Master PIO using MQDMA Bypass Mode Design Example Yes Yes No
Bursting Slave PIO using MQDMA Bypass Mode Design Example Yes Yes No
BAM+BAS PIO using MQDMA Bypass Mode Design Example Yes Yes No
Data Mover Only PIO using MQDMA Bypass Mode Design Example Yes No No
Multi channel DMA Avalon 1-port Device-side Packet Loopback Design Example Yes Yes, 256 channels Yes, support 4 PFs, 64 channel per PF
Multi channel DMA Avalon 1-port Device-side Packet Loopback with SRIOV Design Example Yes Yes, 256 channels No
Multi channel DMA Avalon1-port Packet Generate/ Check Design Example Yes Yes, 256 channels No
Multi channel DMA Avalon 1-port Packet Generate/ Check with SR-IOV Design Example Yes No No
BAM+BAS Traffic Generator/Checker Design Example Yes Yes No
Data Mover Only External Descriptor Controller Design Example Yes No No
Level Two Title