Low Latency 100G Ethernet Intel® FPGA IP Core User Guide: For Intel® Stratix® 10 Devices

Download PDF
ID 683100
Date 2/16/2022
Public

A newer version of this document is available. Customers should click here to go to the newest version.

Document Table of Contents
Document Table of Contents x
1. Low Latency 100G Ethernet Intel FPGA IP Overview 2. Getting Started 3. IP Core Parameters 4. Functional Description 5. Reset 6. Interfaces and Signal Descriptions 7. Registers 8. Debugging the Link 9. Ethernet Toolkit Overview 10. Low Latency 100G Ethernet Intel FPGA IP Core User Guide Archives 11. Document Revision History for the Low Latency 100G Ethernet Intel FPGA IP Core User Guide
1. Low Latency 100G Ethernet Intel FPGA IP Overview x
1.1. Low Latency 100G Ethernet Intel FPGA IP Core Supported Features 1.2. IP Core Device Family and Speed Grade Support 1.3. IP Core Verification 1.4. Performance and Resource Utilization 1.5. Release Information
1.2. IP Core Device Family and Speed Grade Support x
1.2.1. Low Latency 100G Ethernet Intel FPGA IP Core Device Family Support 1.2.2. Low Latency 100G Ethernet Intel FPGA IP Core Device Speed Grade Support
1.3. IP Core Verification x
1.3.1. Simulation Environment 1.3.2. Compilation Checking 1.3.3. Hardware Testing
2. Getting Started x
2.1. Installing and Licensing Intel® FPGA IP Cores 2.2. Specifying the IP Core Parameters and Options 2.3. Simulating the IP Core 2.4. Generated File Structure 2.5. Integrating Your IP Core in Your Design 2.6. IP Core Testbenches 2.7. Compiling the Full Design and Programming the FPGA
2.5. Integrating Your IP Core in Your Design x
2.5.1. Pin Assignments 2.5.2. Adding the Transceiver PLLs 2.5.3. Placement Settings for the Low Latency 100G Ethernet Intel FPGA IP Core
4. Functional Description x
4.1. High Level System Overview 4.2. Low Latency 100G Ethernet Intel FPGA IP Core TX Datapath 4.3. Low Latency 100G Ethernet Intel FPGA IP Core RX Datapath 4.4. Flow Control 4.5. User Interface to Ethernet Transmission 4.6. Auto Negotiation and Link Training 4.7. Auto Adaptation
4.2. Low Latency 100G Ethernet Intel FPGA IP Core TX Datapath x
4.2.1. Preamble, Start, and SFD Insertion 4.2.2. Length/Type Field Processing 4.2.3. Frame Padding 4.2.4. Frame Check Sequence (CRC-32) Insertion 4.2.5. Inter-Packet Gap Adjustment 4.2.6. Error Insertion Test and Debug Feature 4.2.7. TX PCS 4.2.8. TX RSFEC
4.3. Low Latency 100G Ethernet Intel FPGA IP Core RX Datapath x
4.3.1. Low Latency 100G Ethernet Intel FPGA IP Core Preamble Processing 4.3.2. IP Core Strict SFD Checking 4.3.3. Low Latency 100G Ethernet Intel FPGA IP Core FCS (CRC-32) Removal 4.3.4. Low Latency 100G Ethernet Intel FPGA IP Core CRC Checking 4.3.5. Low Latency 100G Ethernet Intel FPGA IP Core Malformed Packet Handling 4.3.6. RX CRC Forwarding 4.3.7. Inter-Packet Gap 4.3.8. RX PCS 4.3.9. RX RSFEC
4.4. Flow Control x
4.4.1. TX Pause/PFC Flow Control Transmission 4.4.2. XON/XOFF Pause Frames
4.5. User Interface to Ethernet Transmission x
4.5.1. Order of Transmission 4.5.2. Bit Order For TX and RX Datapaths
6. Interfaces and Signal Descriptions x
6.1. TX MAC Interface to User Logic 6.2. RX MAC Interface to User Logic 6.3. Transceivers 6.4. Transceiver Reconfiguration Signals 6.5. Avalon-MM Management Interface 6.6. Miscellaneous Status and Debug Signals 6.7. Reset Signals 6.8. Clocks 6.9. Flow Control Interface
6.4. Transceiver Reconfiguration Signals x
6.4.1. Disabling Background Calibration
7. Registers x
7.1. PHY Registers 7.2. TX MAC Registers 7.3. RX MAC Registers 7.4. Pause/PFC Flow Control Registers 7.5. Statistics Registers 7.6. TX Reed-Solomon FEC Registers 7.7. RX Reed-Solomon FEC Registers 7.8. Auto-Negotiation and Link Training Registers
7.5. Statistics Registers x
7.5.1. TX Statistics Registers 7.5.2. RX Statistics Registers
7.8. Auto-Negotiation and Link Training Registers x
7.8.1. AN/LT Sequencer Config 7.8.2. AN/LT Sequencer Status 7.8.3. Auto Negotiation Config Register 1 7.8.4. Auto Negotiation Config Register 2 7.8.5. Auto Negotiation Status Register 7.8.6. Auto Negotiation Config Register 3 7.8.7. Auto Negotiation Config Register 4 7.8.8. Auto Negotiation Config Register 5 7.8.9. Auto Negotiation Config Register 6 7.8.10. Auto Negotiation Status Register 1 7.8.11. Auto Negotiation Status Register 2 7.8.12. Auto Negotiation Status Register 3 7.8.13. Auto Negotiation Status Register 4 7.8.14. Auto Negotiation Status Register 5 7.8.15. Link Training Config Register 1 7.8.16. Link Training Config Register 2 7.8.17. Link Training Status Register 1 7.8.18. Link Training Config Register for Lane 0 7.8.19. Link Training Frame Contents for Lane 0 7.8.20. Local Transceiver TX EQ 1 Settings for Lane 0 7.8.21. Local Transceiver TX EQ 2 Settings for Lane 0 7.8.22. Local Link Training Parameters 7.8.23. Link Training Config Register for Lane 1 7.8.24. Link Training Frame Contents for Lane 1 7.8.25. Local Transceiver TX EQ 1 Settings for Lane 1 7.8.26. Local Transceiver TX EQ 2 Settings for Lane 1 7.8.27. Link Training Config Register for Lane 2 7.8.28. Link Training Frame Contents for Lane 2 7.8.29. Local Transceiver TX EQ 1 Settings for Lane 2 7.8.30. Local Transceiver TX EQ 2 Settings for Lane 2 7.8.31. Link Training Config Register for Lane 3 7.8.32. Link Training Frame Contents for Lane 3 7.8.33. Local Transceiver TX EQ 1 Settings for Lane 3 7.8.34. Local Transceiver TX EQ 2 Settings for Lane 3
9. Ethernet Toolkit Overview x
9.1. Features
1. Low Latency 100G Ethernet Intel FPGA IP Overview
2. Getting Started
3. IP Core Parameters
4. Functional Description
5. Reset
6. Interfaces and Signal Descriptions
7. Registers
8. Debugging the Link
9. Ethernet Toolkit Overview
10. Low Latency 100G Ethernet Intel FPGA IP Core User Guide Archives
11. Document Revision History for the Low Latency 100G Ethernet Intel FPGA IP Core User Guide

8. Debugging the Link

Use the Ethernet Link Inspector (ELI) tool to debug your link.

The ELI is an inspection tool that can continuously monitor an Ethernet link that contains an Ethernet IP, which includes Ethernet lane alignment status, clock data recovery (CDR) lock, media access controller (MAC) statistics, Forward Error Correction (FEC) statistics, and others. If needed, the ELI can capture an event with the help of Signal Tap Logic Analyzer to further examine the link behavior during Auto-negotiation (AN), Link Training (LT), or any other event during the link operation. The ELI also creates a graphical user interface (GUI) to represent the link behavior and is available in the Intel® Quartus® Prime Pro software.

To use ELI, turn on Enable JTAG to Avalon Master Bridge feature in the IP, For more information, refer to the IP Core Parameters.

Begin debugging your link at the most basic level, with word lock. Then, consider higher level issues.

The following steps should help you identify and resolve common problems that occur when bringing up a Low Latency 100G Ethernet Intel FPGA IP core link:

  1. Establish word lock—The RX lanes should be able to achieve word lock even in the presence of extreme bit error rates. If the IP core is unable to achieve word lock, check the transceiver clocking and data rate configuration. Check for cabling errors such as the reversal of the TX and RX lanes. Check the clock frequency monitors in the Control and Status registers.

    To check for word lock: Clear the FRM_ERR register by writing the value of 1 followed by another write of 0 to the SCLR_FRM_ERR register at offset 0x324.Then read the FRM_ERR register at offset 0x323. If the value is zero, the core has word lock. If non-zero the status is indeterminate.

  2. When having problems with word lock, check the EIO_FREQ_LOCK register at address 0x321. The values in this register define the status of the recovered clock. In normal operation, all the bits should be asserted. A non-asserted (value-0) or toggling logic value on the bit that corresponds to any lane, indicates a clock recovery problem. Clock recovery difficulties are typically caused by the following problems:
    • A high bit error rate (BER)
    • Failure to establish the link
    • Incorrect clock inputs to the IP core
  3. Check the PMA FIFO levels by selecting appropriate bits in the EIO_FLAG_SEL register and reading the values in the EIO_FLAGS register. During normal operation, the TX and RX FIFOs should be nominally filled. Observing a the TX FIFO is either empty or full typically indicates a problem with clock frequencies. The RX FIFO should never be full, although an empty RX FIFO can be tolerated.
  4. Establish lane integrity—When operating properly, the lanes should not experience bit errors at a rate greater than roughly one per day. Bit errors within data packets are identified as FCS errors. Bit errors in control information, including IDLE frames, generally cause errors in XL/CGMII decoding.
  5. If the IP core acquires word lock but the link is still not established, check the AM_LOCK register at offset 0x328 by reading it repeatedly. If it is deasserted or toggling, check the cables and connections.
  6. If the IP core acquires alignment marker lock on all virtual lanes (bit [0] of the AM_LOCK has the consistent value of 1), but the link is still not established, check the LANE_DESKEWED register at offset 0x329. If this register remains at the value of 0, the skew is greater than the deskew limit.
  7. Verify packet traffic—The Ethernet protocol includes automatic lane reordering so the higher levels should follow the PCS. If the PCS is locked, but higher level traffic is corrupted, there may be a problem with the remote transmitter virtual lane tags.
  8. Tuning—You can adjust analog parameters to improve the bit error rate. IDLE traffic is representative for analog purposes.

In addition, your IP core can experience loss of signal on the Ethernet link after it is established. In this case, the TX functionality is unaffected, but the RX functionality is disrupted. The following symptoms indicate a loss of signal on the Ethernet link:

  • The IP core deasserts the rx_pcs_ready signal, indicating the IP core has lost alignment marker lock.
  • The IP core deasserts the RX PCS fully aligned status bit (bit [0]) of the RX_PCS_FULLY_ALIGNED_S register at offset 0x326. This change is linked to the change in value of the rx_pcs_ready signal.
  • If Enable link fault generation is turned on, the IP core sets local_fault_status to the value of 1.
  • The IP core asserts the Local Fault Status bit (bit [0]) of the Link_Fault register at offset 0x508. This change is linked to the change in value of the local_fault_status signal.
  • The IP core triggers the RX digital reset process by asserting soft_rxp_rst.
Related Information
Level Two Title