Multi Channel DMA Intel® FPGA IP for PCI Express* User Guide

Download PDF
ID 683821
Date 10/06/2023
Public

A newer version of this document is available. Customers should click here to go to the newest version.

Document Table of Contents
Document Table of Contents x
1. Before You Begin 2. Introduction 3. Functional Description 4. Interface Overview 5. Parameters (H-Tile) 6. Parameters (P-Tile) (F-Tile) (R-Tile) 7. Designing with the IP Core 8. Software Programming Model 9. Registers 10. Troubleshooting/Debugging 11. Multi Channel DMA Intel FPGA IP for PCI Express User Guide Archives 12. Revision History for the Multi Channel DMA Intel FPGA IP for PCI Express User Guide
1. Before You Begin x
1.1. Terms and Acronyms 1.2. Known Issues
2. Introduction x
2.1. Multi Channel DMA IP for PCI Express Features 2.2. Device Family Support 2.3. Recommended Speed Grades 2.4. Resource Utilization 2.5. Release Information
2.1. Multi Channel DMA IP for PCI Express Features x
2.1.1. Endpoint Mode 2.1.2. Root Port Mode
3. Functional Description x
3.1. Multi Channel DMA 3.2. Bursting Avalon-MM Master (BAM) 3.3. Bursting Avalon-MM Slave (BAS) 3.4. MSI Interrupt 3.5. Config Slave (CS) 3.6. Root Port Address Translation Table Enablement 3.7. Hard IP Reconfiguration Interface 3.8. Config TL Interface 3.9. Configuration Intercept Interface (EP Only) 3.10. Data Mover Only
3.1. Multi Channel DMA x
3.1.1. H2D Data Mover 3.1.2. D2H Data Mover 3.1.3. Descriptors 3.1.4. Avalon-MM PIO Master 3.1.5. Avalon-MM Write (H2D) and Read (D2H) Master 3.1.6. Avalon-ST Source (H2D) and Sink (D2H) 3.1.7. User MSI-X 3.1.8. User Functional Level Reset (FLR) 3.1.9. Control Registers
3.1.2. D2H Data Mover x
3.1.2.1. D2H Descriptor Fetch
3.1.3. Descriptors x
3.1.3.1. Support for Unaligned (or byte-aligned) Data Transfer 3.1.3.2. Metadata Support 3.1.3.3. MSI-X/Writeback
3.1.6. Avalon-ST Source (H2D) and Sink (D2H) x
3.1.6.1. Avalon-ST 1-Port Mode 3.1.6.2. Packet (File) Boundary 3.1.6.3. Metadata
3.4. MSI Interrupt x
3.4.1. Endpoint MSI Support through BAS 3.4.2. MSI Interrupt Controller
3.5. Config Slave (CS) x
3.5.1. 29 bit AVMM address format 3.5.2. 14 bit AVMM address format 3.5.3. Configuration Access Mechanism
3.10. Data Mover Only x
3.10.1. H2D Data Mover 3.10.2. D2H Data Mover 3.10.3. Application Specific Bits
3.10.1. H2D Data Mover x
3.10.1.1. H2D Descriptor Format (h2ddm_desc) 3.10.1.2. H2D Descriptor Completion Packet Format (h2ddm_desc_cmpl) 3.10.1.3. H2D Descriptor Status (h2ddm_desc_status)
3.10.2. D2H Data Mover x
3.10.2.1. D2H Descriptor Format (d2hdm_desc) 3.10.2.2. D2H Descriptor Status (d2hdm_desc_status)
4. Interface Overview x
4.1. Port List 4.2. Clocks 4.3. Resets 4.4. Multi Channel DMA 4.5. Bursting Avalon-MM Master (BAM) Interface 4.6. Bursting Avalon-MM Slave (BAS) Interface 4.7. MSI Interface 4.8. Config Slave Interface (RP only) 4.9. Hard IP Reconfiguration Interface 4.10. Config TL Interface 4.11. Configuration Intercept Interface (EP Only) 4.12. Data Mover Interface 4.13. Hard IP Status Interface
4.1. Port List x
4.1.1. Port List (H-Tile) 4.1.2. Port List (P-Tile) (F-Tile) (R-Tile)
4.2. Clocks x
4.2.1. F-Tile System PLL Reference Clock Requirements
4.4. Multi Channel DMA x
4.4.1. Avalon-MM PIO Master 4.4.2. Avalon-MM Write Master (H2D) 4.4.3. Avalon-MM Read Master (D2H) 4.4.4. Avalon-ST Source (H2D) 4.4.5. Avalon-ST Sink (D2H) 4.4.6. User Event MSI-X Interface 4.4.7. User Functional Level Reset (FLR) Interface
4.12. Data Mover Interface x
4.12.1. H2D Data Mover Interface 4.12.2. D2H Data Mover Interface
5. Parameters (H-Tile) x
5.1. IP Settings 5.2. Example Designs
5.1. IP Settings x
5.1.1. System Settings 5.1.2. MCDMA Settings 5.1.3. Device Identification Registers 5.1.4. Multifunction and SR-IOV System Settings Parameters [Endpoint Mode] 5.1.5. Configuration, Debug and Extension Options 5.1.6. PHY Characteristics 5.1.7. PCI Express / PCI Capabilities Parameters
5.1.7. PCI Express / PCI Capabilities Parameters x
5.1.7.1. Device 5.1.7.2. Link 5.1.7.3. MSI-X 5.1.7.4. Power Management 5.1.7.5. Vendor Specific Extended Capability (VSEC)
6. Parameters (P-Tile) (F-Tile) (R-Tile) x
6.1. Top-Level Settings 6.2. PCIe0 Settings 6.3. Example Designs 6.4. Analog Parameters (F-Tile MCDMA IP Only) 6.5. PCIe1 Settings
6.2. PCIe0 Settings x
6.2.1. Base Address Register 6.2.2. PCIe0 Configuration, Debug and Extension Options 6.2.3. PCIe0 Device Identification Registers 6.2.4. PCIe0 PCI Express / PCI Capabilities 6.2.5. MCDMA Settings
6.2.3. PCIe0 Device Identification Registers x
6.2.3.1. PCIe0 PF0 IDs 6.2.3.2. PCIe0 PF0 VF IDs
6.2.4. PCIe0 PCI Express / PCI Capabilities x
6.2.4.1. PCIe0 Device 6.2.4.2. PCIe0 Link 6.2.4.3. PCIe0 MSI-X 6.2.4.4. PCIe0 DEV SER 6.2.4.5. PCIe0 PRS 6.2.4.6. PCIe0 PTM 6.2.4.7. PCIe0 ACS Capabilities 6.2.4.8. PCIe0 ATS 6.2.4.9. PCIe0 TPH
6.2.4.7. PCIe0 ACS Capabilities x
6.2.4.7.1. PCIe0 ACS for Physical Functions 6.2.4.7.2. PCIe0 ACS for Virtual Functions
6.2.4.8. PCIe0 ATS x
6.2.4.8.1. PCIe0 ATS for Physical Functions 6.2.4.8.2. PCIe0 ATS for Virtual Functions
6.2.4.9. PCIe0 TPH x
6.2.4.9.1. PCIe0 TPH for Physical Functions 6.2.4.9.2. PCIe0 TPH for Virtual Functions
6.5. PCIe1 Settings x
6.5.1. PCIe1 Configuration, Debug and Extension Options
7. Designing with the IP Core x
7.1. Generating the IP Core 7.2. Simulating the IP Core 7.3. IP Core Generation Output - Intel® Quartus® Prime Pro Edition 7.4. Systems Integration and Implementation
7.4. Systems Integration and Implementation x
7.4.1. Required Supporting IP
8. Software Programming Model x
8.1. Multi Channel DMA IP Custom Driver 8.2. Multi Channel DMA IP DPDK Poll-Mode based Driver 8.3. Multi Channel DMA IP Kernel Mode Network Device Driver
8.1. Multi Channel DMA IP Custom Driver x
8.1.1. Architecture 8.1.2. libmqdma library details 8.1.3. Application 8.1.4. Software Flow 8.1.5. API Flow 8.1.6. libmqdma Library API List 8.1.7. Request Structures
8.1.2. libmqdma library details x
8.1.2.1. Channel Initialization 8.1.2.2. Descriptor Memory Management 8.1.2.3. Descriptor Completion Status Update
8.1.5. API Flow x
8.1.5.1. Single Descriptor Load and Submit 8.1.5.2. Multiple Descriptor Load and Submit
8.1.6. libmqdma Library API List x
8.1.6.1. ifc_api_start 8.1.6.2. ifc_mcdma_port_by_name 8.1.6.3. ifc_qdma_device_get 8.1.6.4. ifc_num_channels_get 8.1.6.5. ifc_qdma_channel_get 8.1.6.6. ifc_qdma_acquire_channels 8.1.6.7. ifc_qdma_release_all_channels 8.1.6.8. ifc_qdma_device_put 8.1.6.9. ifc_qdma_channel_put 8.1.6.10. ifc_qdma_completion_poll 8.1.6.11. ifc_qdma_request_start 8.1.6.12. ifc_qdma_request_prepare 8.1.6.13. ifc_qdma_descq_queue_batch_load 8.1.6.14. ifc_qdma_request_submit 8.1.6.15. ifc_qdma_pio_read32 8.1.6.16. ifc_qdma_pio_write32 8.1.6.17. ifc_qdma_pio_read64 8.1.6.18. ifc_qdma_pio_write64 8.1.6.19. ifc_qdma_pio_read128 8.1.6.20. ifc_qdma_pio_write128 8.1.6.21. ifc_qdma_pio_read256 8.1.6.22. ifc_qdma_pio_write256 8.1.6.23. ifc_request_malloc 8.1.6.24. ifc_request_free 8.1.6.25. ifc_app_stop 8.1.6.26. ifc_qdma_poll_init 8.1.6.27. ifc_qdma_poll_add 8.1.6.28. ifc_qdma_poll_wait 8.1.6.29. ifc_mcdma_port_by_name
8.2. Multi Channel DMA IP DPDK Poll-Mode based Driver x
8.2.1. Architecture 8.2.2. MCDMA Poll Mode Driver 8.2.3. DPDK based application 8.2.4. Request Structures 8.2.5. Software Flow 8.2.6. API Flow 8.2.7. API List
8.2.2. MCDMA Poll Mode Driver x
8.2.2.1. Completion Status Update 8.2.2.2. Metadata Support 8.2.2.3. User MSI-X
8.3. Multi Channel DMA IP Kernel Mode Network Device Driver x
8.3.1. Architecture 8.3.2. Driver Information 8.3.3. Software Flow
8.3.2. Driver Information x
8.3.2.1. Device Management 8.3.2.2. Channel Management 8.3.2.3. Descriptor Memory Management 8.3.2.4. Completions Management 8.3.2.5. ethtool support 8.3.2.6. debugfs support 8.3.2.7. SRIOV Support
8.3.3. Software Flow x
8.3.3.1. Host to Device Flow 8.3.3.2. Device to Host Flow
9. Registers x
9.1. Queue Control (QCSR) 9.2. MSI-X Memory Space 9.3. Control Register (GCSR)
10. Troubleshooting/Debugging x
10.1. Debug Toolkit
10.1. Debug Toolkit x
10.1.1. Overview 10.1.2. Enabling the Debug Toolkit 10.1.3. Launching the Debug Toolkit 10.1.4. Using the Debug Toolkit
10.1.4. Using the Debug Toolkit x
10.1.4.1. Main View 10.1.4.2. Toolkit Parameters 10.1.4.3. Channel Parameters 10.1.4.4. Eye Viewer 10.1.4.5. Link Inspector
10.1.4.4. Eye Viewer x
10.1.4.4.1. Running Eye Viewer in the P-Tile Debug Toolkit 10.1.4.4.2. Running Eye Viewer in the F-Tile Debug Toolkit
1. Before You Begin
2. Introduction
3. Functional Description
4. Interface Overview
5. Parameters (H-Tile)
6. Parameters (P-Tile) (F-Tile) (R-Tile)
7. Designing with the IP Core
8. Software Programming Model
9. Registers
10. Troubleshooting/Debugging
11. Multi Channel DMA Intel FPGA IP for PCI Express User Guide Archives
12. Revision History for the Multi Channel DMA Intel FPGA IP for PCI Express User Guide

2.4. Resource Utilization

Table 5.  Intel Stratix 10 H-Tile and P-Tile PCIe x16 [Avalon-MM Interface]
User Mode Link Conf DMA Channels ALMs Logic Registers M20Ks
H-Tile P-Tile H-Tile P-Tile H-Tile P-Tile
MCDMA

PCIe Gen3 x16 for H-Tile

PCIe Gen4 x16 for P-Tile

 256 44,034 37,502 109,399 99,491 532 512
BAM_MCDMA

PCIe Gen4 x16 for P-Tile

PCIe Gen3 x16 for H-Tile

256 48,447 41,835 120,555 110,600 616 596

BAM_BAS_MCDMA

PCIe Gen3 x16 for H-Tile

 2048 52.1296 - 143,217 - 858 -
BAM

PCIe Gen4 x16 for P-Tile

PCIe Gen3 x16 for H-Tile

n/a 25,162 17,567 53,976 42,111 307 285
BAS

PCIe Gen4 x16 for P-Tile

PCIe Gen3 x16 for H-Tile

n/a 26,818 20,126 61,369 49,486 257 236
BAM+BAS

PCIe Gen4 x16 for P-Tile

PCIe Gen3 x16 for H-Tile

n/a 33,655 25,104 78,809 65,025 372 346
Table 6.  Intel Stratix 10 H-Tile and P-Tile PCIe x8 [Avalon-MM Interface]
User Mode Link Conf DMA Channels ALMs Logic Registers M20Ks
H-Tile P-Tile H-Tile P-Tile H-Tile P-Tile
MCDMA

PCIe Gen3 x8 for H-Tile

PCIe Gen4 x8 for P-Tile

 256 22,914 25,822 61,888 69,774 397 372
BAM_MCDMA

PCIe Gen3 x8 for H-Tile

PCIe Gen4 x8 for P-Tile

 256 25,329 28,320 68,691 76,285 452 431

BAM_BAS_MCDMA

PCIe Gen3 x8 for H-Tile

2048

 33956.2 - 97,495 - 544 -
BAM

PCIe Gen3 x8 for H-Tile

PCIe Gen4 x8 for P-Tile

 n/a 8,257 9,938 21,171 27,441 199 177
BAS

PCIe Gen3 x8 for H-Tile

PCIe Gen4 x8 for P-Tile

 n/a 9,227 11,374 24,973 31,260 169 149
BAM+BAS

PCIe Gen3 x8 for H-Tile

PCIe Gen4 x8 for P-Tile

 n/a 12,530 14,563 34,508 40,592 248 226
Table 7.  Intel Stratix 10 H-Tile and P-Tile PCIe x16 [1 port Avalon-ST]
User Mode Link Conf DMA Channels ALMs Logic Registers M20Ks
H-Tile P-Tile H-Tile P-Tile H-Tile P-Tile
MCDMA

PCIe Gen3 x16 for H-Tile

PCIe Gen4 x16 for P-Tile

 1 / 32 /64 47866/ 50,093/ 52,951 38,634/ 41,181/ 43,852 117470/ 122,854/ 128,771 104793/ 110,305/ 115,833 560/ 578/ 601 536/ 555/ 576
BAM_MCDMA

PCIe Gen3 x16 for H-Tile

PCIe Gen4 x16 for P-Tile

 2 / 32 /64 51976/ 54,300/ 57,132 42,155 / 43,745/ 45,118 128208/ 133,935/ 139,874 113,660 / 117,292/ 120,406 643/ 662/ 684 615 / 625/ 638
Table 8.   Intel Agilex® 7 P-Tile and F-Tile PCIe x16 [Avalon-MM Interface]
User Mode Link Conf DMA Channels ALMs Logic Registers M20Ks
P-Tile F-Tile P-Tile F-Tile P-Tile F-Tile
MCDMA Gen4 x16 256 33,805 37,445 97,557 103,143 512 521
BAM_MCDMA Gen4 x16 256 38,546 42,198 108,328 113,886 595 605

BAM_BAS_MCDMA

Gen4 x16

 2048 43,907 43999.4 139591 139552 855 855
BAM Gen4 x16 n/a 17,246 20,780 42,097 47,680 285 295
BAS Gen4 x16 n/a 19,164 22,677 49,327 54,854 236 246
BAM+BAS Gen4 x16 n/a 24,955 28,562 64,885 70,342 346 356
Table 9.   Intel Agilex® 7 P-Tile and F-Tile PCIe x8 [Avalon-MM Interface]
User Mode Link Conf DMA Channels ALMs Logic Registers M20Ks
P-Tile F-Tile P-Tile F-Tile P-Tile F-Tile
MCDMA Gen4 x8 256 22,254 23,864 67,551 69,063 372 383
BAM_MCDMA Gen4 x8 256 24,440 26,085 74,195 75,716 431 441

BAM_BAS_MCDMA

Gen4 x8

 2048 27519.4 27606.4 92671 92728 539 539
BAM Gen4 x8 n/a 9,052 10,689 27,189 28,675 177 187
BAS Gen4 x8 n/a 10,331 11,907 31,029 32,514 149 159
BAM+BAS Gen4 x8 n/a 13,319 14,933 40,518 41,988 226 236
Table 10.   Intel Agilex® 7 R-Tile PCIe x8 [Avalon-MM Interface]
User Mode Link Conf DMA Channels ALMs Logic Registers M20Ks

MCDMA

 Gen5 x8 2048

36359.8

115,373

857

BAM

 Gen5 x8 -

15022.4

45,060

327

BAM+BAS

 Gen5 x8 -

14932.2

45,036

327

BAM_MCDMA

 Gen5 x8 2048

35936.5

113,178

843

Table 11.   Intel Agilex® 7 P-Tile and F-Tile PCIe x16 [1 port Avalon-ST]
User Mode Link Conf DMA Channels ALMs Logic Registers M20Ks
P-Tile F-Tile P-Tile F-Tile P-Tile F-Tile
MCDMA Gen4 x16 1 / 32 /64 33,913/ 36,373/ 39,480 37,567 / 40,071 / 43,078 102712/ 108,215/ 114,039 108,303 / 113,764 / 119,553 537/ 554/ 576 546 / 564 / 587
BAM_MCDMA Gen4 x16 2 / 32 /64 38,247 / 39,448/ 41,041 41,880 / 43,115 / 44,686 112,445 / 115,445 / 118,806 118,007 / 120,995 / 124,434 620 / 625 / 639 629 / 636 / 648
Table 12.   Intel Agilex® 7 P-Tile and F-Tile PCIe x8 [1 port Avalon-ST]
User Mode Link Conf DMA Channels ALMs Logic Registers M20Ks
P-Tile F-Tile P-Tile F-Tile P-Tile F-Tile
MCDMA Gen4 x8 1 / 32 /64 22,978 / 25,343/ 28,399 24,705 / 27,066 / 30,219 72,007/ 77,499/ 83,182 73,565 / 79,005 / 84,731 397 / 413 / 436 407 / 424 / 446
BAM_MCDMA Gen4 x8 2 / 32 /64 24,790 / 26,083/ 27,550 26,541 / 27,776 / 29,334 77,532 / 80,585 / 84,057 79,104 / 82,126 / 85,545 455 / 461 / 473 465 / 470 / 483
Table 13.   Intel Agilex® 7 P-Tile and F-Tile Data Mover Only User Mode
User Mode Link Conf DMA Channels ALMs Logic Registers M20Ks
P-Tile F-Tile P-Tile F-Tile P-Tile F-Tile
Data Mover Only Gen4 x16 n/a 31,528 41,514 83,773 91,219 522 532

Data Mover Only

 Gen4 x8 n/a 18,163 25,718 56,890 60,391 381 391
Table 14.  Intel Stratix 10 P-Tile Data Mover Only User Mode
User Mode Link Configuration DMA Channels ALMs Logic Registers M20Ks
Data Mover Only Gen4 x16 n/a 33,214 84,200 522

Data Mover Only

 Gen4 x8 n/a 21,149 57,077 381
Level Two Title