F-Tile Ethernet Intel® FPGA Hard IP User Guide

Download PDF
ID 683023
Date 1/07/2022
Public

A newer version of this document is available. Customers should click here to go to the newest version.

Document Table of Contents
Document Table of Contents x
1. Overview 2. Getting Started 3. F-Tile Ethernet Intel® FPGA Hard IP Parameters 4. Functional Description 5. Clocks 6. Resets 7. Interface Overview 8. Configuration Registers 9. Supported Modules and IPs 10. Supported Tools 11. F-Tile Ethernet Intel® FPGA Hard IP User Guide Archives 12. Document Revision History
1. Overview x
1.1. Ethernet System in F-Tile Overview 1.2. F-Tile Ethernet Intel® FPGA Hard IP Overview
1.2. F-Tile Ethernet Intel® FPGA Hard IP Overview x
1.2.1. Device Family Support 1.2.2. Device Speed Grade Support 1.2.3. Resource Utilization 1.2.4. Release Information
2. Getting Started x
2.1. Installing and Licensing F-Tile Ethernet Intel® FPGA Hard IP Cores 2.2. Specifying the IP Core Parameters and Options 2.3. Reference and System PLL Clock for your IP Design 2.4. Generated File Structure 2.5. Generating Tile Files 2.6. IP Core Testbenches
4. Functional Description x
4.1. Datapath Description 4.2. F-Tile Ethernet Intel® FPGA Hard IP MAC 4.3. PCS, OTN, and FlexE Modes 4.4. Precision Time Protocol 4.5. Auto-Negotiation and Link Training
4.2. F-Tile Ethernet Intel® FPGA Hard IP MAC x
4.2.1. MAC TX Datapath 4.2.2. MAC RX Datapath 4.2.3. Congestion and Flow Control Using PAUSE or Priority Flow Control (PFC) 4.2.4. Link Fault Signaling 4.2.5. Order of Ethernet Transmission
4.2.1. MAC TX Datapath x
4.2.1.1. TX Preamble, Start, and SFD Insertion 4.2.1.2. Source Address Insertion 4.2.1.3. Length/Type Field Processing 4.2.1.4. Frame Padding 4.2.1.5. Frame Check Sequence (CRC-32) Insertion 4.2.1.6. Inter-Packet Gap Generation and Insertion
4.2.2. MAC RX Datapath x
4.2.2.1. RX Preamble Processing 4.2.2.2. RX Strict SFD Checking 4.2.2.3. RX FCS Checking 4.2.2.4. RX Malformed Packet Handling 4.2.2.5. Removing PAD Bytes and FCS Bytes from RX Frames 4.2.2.6. RX Undersized Frames, Oversized Frames, and Frames with Length Errors 4.2.2.7. Inter-Packet Gap
4.2.3. Congestion and Flow Control Using PAUSE or Priority Flow Control (PFC) x
4.2.3.1. Conditions Triggering XOFF Frame Transmission 4.2.3.2. Conditions Triggering XON Frame Transmission 4.2.3.3. Pause Control and Generation Interface 4.2.3.4. Pause Control Frame Filtering
4.3. PCS, OTN, and FlexE Modes x
4.3.1. PCS Mode 4.3.2. OTN Mode 4.3.3. FlexE Mode
4.4. Precision Time Protocol x
4.4.1. Features 4.4.2. PTP Timestamp Accuracy 4.4.3. PTP Client Flow 4.4.4. RX Virtual Lane Offset Calculation for No FEC Variants 4.4.5. Virtual Lane Order and Offset Values 4.4.6. UI Adjustment 4.4.7. Reference Time Interval 4.4.8. Minimum and Maximum Reference Time (TAM) Interval for UI Measurement (Hardware) 4.4.9. UI Value and PMA Delay 4.4.10. Routing Delay Adjustment for Advanced Timestamp Accuracy Mode
4.4.3. PTP Client Flow x
4.4.3.1. PTP TX Client Flow 4.4.3.2. PTP RX Client Flow
4.4.6. UI Adjustment x
4.4.6.1. TX UI Adjustment 4.4.6.2. RX UI Adjustment
5. Clocks x
5.1. Clock Connections in Single Instance Operation 5.2. Clock Connections in Multiple Instance Operation 5.3. Clock Connections in MAC Asynchronous FIFO Operation 5.4. Clock Connections in PTP-Based Synchronous and Asynchronous Operation 5.5. Clock Connections in Synchronous Ethernet Operation 5.6. Custom Cadence
6. Resets x
6.1. Reset Signals 6.2. Reset Sequence
7. Interface Overview x
7.1. Status Interface 7.2. TX MAC Avalon ST Client Interface 7.3. RX MAC Avalon ST Aligned Client Interface 7.4. TX MAC Segmented Client Interface 7.5. RX MAC Segmented Client Interface 7.6. MAC Flow Control Interface 7.7. PCS Mode TX Interface 7.8. PCS Mode RX Interface 7.9. FlexE and OTN Mode TX Interface 7.10. FlexE and OTN Mode RX Interface 7.11. Custom Rate Interface 7.12. Reconfiguration Interfaces 7.13. Precision Time Protocol Interface
7.2. TX MAC Avalon ST Client Interface x
7.2.1. TX MAC Avalon ST Client Interface with Disabled Preamble Passthrough 7.2.2. TX MAC Avalon ST Client Interface with Enabled Preamble Passthrough 7.2.3. Using MAC Avalon ST skip_crc Signal to Control Source Address, PAD, and CRC Insertion 7.2.4. Using MAC Avalon ST i_tx_error Signal to Mark Packets Invalid
7.3. RX MAC Avalon ST Aligned Client Interface x
7.3.1. RX MAC Avalon ST Client Interface with Disabled Preamble Passthrough 7.3.2. RX MAC Avalon ST Client Interface with Enabled Passthrough and RX CRC Forwarding 7.3.3. RX MAC Adapter Limitations 7.3.4. RX MAC Avalon ST Client Interface Status
7.4. TX MAC Segmented Client Interface x
7.4.1. TX MAC Segmented Client Interface with Disabled Preamble Passthrough 7.4.2. TX MAC Segmented Client Interface with Enabled Preamble Passthrough 7.4.3. Using MAC Segmented skip_crc Signal to Control Source Address, PAD, and CRC Insertion 7.4.4. Using MAC Segmented i_tx_mac_error to Mark Packets Invalid
7.5. RX MAC Segmented Client Interface x
7.5.1. RX MAC Segmented Client Interface with Disabled Preamble Passthrough 7.5.2. RX MAC Segmented Client Interface with Enabled Passthrough and RX CRC Forwarding 7.5.3. RX MAC Segmented Client Interface Status and Errors
7.12. Reconfiguration Interfaces x
7.12.1. Ethernet Reconfiguration Interfaces 7.12.2. Transceiver Reconfiguration Interfaces
7.13. Precision Time Protocol Interface x
7.13.1. Time-of-Day Interface 7.13.2. TX 2-Step Timestamp Interface 7.13.3. TX 1-Step Timestamp Interface 7.13.4. RX Timestamp Interface 7.13.5. PTP Status Interface 7.13.6. PTP Tile Interface
7.13.3. TX 1-Step Timestamp Interface x
7.13.3.1. PTP Field Offset for 1-Step Operation
8. Configuration Registers x
8.1. Ethernet Avalon® Memory-Mapped Interface Address Space 8.2. Transceiver Avalon® Memory-Mapped Interface Address Space
8.1. Ethernet Avalon® Memory-Mapped Interface Address Space x
8.1.1. Ethernet Hard IP Core CSRs 8.1.2. FEC and Transceiver Control and Status Registers
9. Supported Modules and IPs x
9.1. F-Tile Auto-Negotiation and Link Training For Ethernet Intel® FPGA IP 9.2. PTP Tile Adapter
9.1. F-Tile Auto-Negotiation and Link Training For Ethernet Intel® FPGA IP x
9.1.1. Overview 9.1.2. Release Information 9.1.3. Functional Description 9.1.4. Parameters 9.1.5. Clocks and Resets 9.1.6. Registers
9.2. PTP Tile Adapter x
9.2.1. Overview 9.2.2. Clocks, Reset, and Interface Ports 9.2.3. PTP Asymmetry Delay Reconfiguration Interface 9.2.4. PTP Peer-to-Peer MeanPathDelay Reconfiguration Interface
10. Supported Tools x
10.1. F-Tile Channel Placement Tool 10.2. Ethernet Toolkit Overview
10.2. Ethernet Toolkit Overview x
10.2.1. Features
1. Overview
2. Getting Started
3. F-Tile Ethernet Intel® FPGA Hard IP Parameters
4. Functional Description
5. Clocks
6. Resets
7. Interface Overview
8. Configuration Registers
9. Supported Modules and IPs
10. Supported Tools
11. F-Tile Ethernet Intel® FPGA Hard IP User Guide Archives
12. Document Revision History

4.4.3.2. PTP RX Client Flow

In this section, the acronyms PL and VL stand for Physical Lane and Virtual Lane respectively.

The following flows depict pseudo-code meant for the conceptual, illustrative purposes. For definitive software routines, refer to the design example.

  1. After power up or RX reset, wait until RX PCS is fully aligned.
    You can monitor the status via one of the following:
    • Output port:
      o_rx_pcs_fully_alligned = 1'b1
    • Polling via Avalon® memory-mapped interface register until it is asserted:
      • For 10GE - 100GE no FEC variants and 25G FEC variants: 
        csr_read(phy_rxpcs_status.rx_aligned) = 1’b1
      • For 50GE - 400GE FEC variants: 
        csr_read(rsfec_aggr_rx_stat.not_align) = 1’b0
  2. For FEC variants, configure RX FEC codeword position into the transceiver.
    Attention: You must skip this step for non-FEC variants.
    1. Read RX FEC codeword position and FEC lane mapping for each PMA lane.

      Determine mapping from a PMA lane to a FEC lane. 

      pl_fl_map = Speed / 25 / PL
where Speed = {25, 50, 100, 200, 400}

for (fl = 0; fl < FL; fl++) {
   rx_fec_cw_pos[fl] = csr_read(rsfec_cw_pos_rx[fl][14:0])
}
    2. Calculate pulse adjustments:
      for (fl = 0; fl < FL; fl++) {
   rx_xcvr_if_pulse_adj[fl] = rx_fec_cw_pos[fl]
}
    3. Write the pulse adjustments into the IP:
      For multiple FEC lanes that interleave within single transceiver, select adjustment value from FEC lane with lowest index. You must ensure correct mapping between PMA lane and the corresponding CSR registers.
      • For FGT transceiver:
        for (pl = 0; pl < PL; pl++) {
   csr_write(ux_q_dl_ctrl_a_l<apl>.cfg_rx_lat_bit_for_async[17:0], 
             rx_xcvr_if_pulse_adj[pl*pl_fl_map])
}
        Note: There are 4 FGT quads with 4 apl lanes each. You must program registers of all active quad lanes. For information on accessing different FGT quads, refer to the User Guide.
      • For FHT transceiver:
        for (pl = 0; pl < PL; pl++) {
   csr_write(rxdl_async_l<apl>.cfg_rx_lat_bit_for_async_lane<pl>[17:0],
             rx_xcvr_if_pulse_adj[pl*pl_fl_map])
}
        Note: Both apl and pl are used in this step. For more information about the physical lanes, refer to Client Flow Glossary table in .
    4. Notify soft PTP that pulse adjustments have been configured.
      csr_write(ptp_rx_user_cfg_status.rx_fec_cw_pos_cfg_done, 1'b1)
  3. Wait until RX raw offset data are ready.
    You can monitor the status via one of the following:
    • Output port:
      o_rx_ptp_offset_data_valid = 1'b1
    • Polling via CSR:
      csr_read(ptp_status.rx_ptp_offset_data_valid) = 1’b1
  4. Read RX raw offset data from IP:
    • All variants:
      rx_const_delay = csr_read(ptp_rx_lane_calc_data_constdelay[30:0])
rx_const_delay_sign = csr_read(ptp_rx_lane_calc_data_constdelay[31])

for (pl = 0; pl < PL; pl++) {
 rx_apulse_offset[pl] = csr_read(ptp_rx_lane<pl>_calc_data_offset[30:0])
 rx_apulse_offset_sign[pl] = csr_read(ptp_rx_lane<pl>_calc_data_offset[31])
 rx_apulse_wdelay[pl] = csr_read(ptp_rx_lane<pl>_calc_data_wiredelay[19:0])
 rx_apulse_time[pl]   = csr_read(ptp_rx_lane<pl>_calc_data_time[27:0])
}
    • 10GE/25GE no FEC variants:
      rx_bitslip_cnt       = csr_read(bitslip_cnt.bitslip_cnt[6:0])
rx_dlpulse_alignment = csr_read(bitslip_cnt.dlpulse_alignment)
  5. Determine RX reference lane.

    Skip steps 5b, 5c, and 5d for single lane 10G and 25G Ethernet modes. For these variants, use: 

    rx_ref_pl = 0
rx_ref_fl = 0
rx_ref_vl = 0
    1. Determine synchronous pulse (Alignment Marker (AM)) offsets with reference to asynchronous pulse.
      • FEC variants: 
        for (fl = 0; fl < FL; fl++) {
if ((rx_xcvr_if_pulse_adj[fl] 
   + rx_xcvr_if_pulse_adj[fl–(fl%pl_fl_map)][4:0]) 
   > rx_xcvr_if_pulse_adj[fl–(fl%pl_fl_map)]) 
{
 rx_spulse_offset[fl] =
   (rx_xcvr_if_pulse_adj[fl] 
  – rx_xcvr_if_pulse_adj[fl–(fl%pl_fl_map)] 
  + rx_xcvr_if_pulse_adj[fl–(fl%pl_fl_map)][4:0]) * UI14 * pl_fl_map

 rx_spulse_offset_sign[fl] = 1’b0;
} else 
{
 rx_spulse_offset[fl] = 
   (rx_xcvr_if_pulse_adj[fl–(fl%pl_fl_map)] 
  – rx_xcvr_if_pulse_adj[fl] 
  – rx_xcvr_if_pulse_adj[fl–(fl%pl_fl_map)][4:0]) * UI14 * pl_fl_map
 rx_spulse_offset_sign[fl] = 1’b1;
}
}
      • 50GE/100GE no FEC variants: 
        for (vl = 0; vl < VL; vl++) {
   rx_spulse_post_am[vl]     = <Refer to RX Virtual Lane Offset Calculation for No FEC Variants>
   rx_spulse_offset[vl]      = (AM_INTERVAL – rx_spulse_post_am[vl]) * UI14
   rx_spulse_offset_sign[vl] = 1’b0;
}
        The offset calculated in RX Virtual Lane Offset Calculation for No FEC Variants measures the bit distance from PCS internal AM to synchronous pulse of each virtual lane. To determine the bit distance from synchronous pulse to the next AM, subtract the calculated value from the AM interval.
        Variants AM INTERVAL in Simulation AM INTERVAL in Hardware
        50GE-2 2,560 * 66 = 168,960 32,768 * 66 = 2,162,688
        100GE-4 2,560 * 66 = 168,960 81,920 * 66 = 5,406,720
      • For 10GE/25GE no FEC variants:
        rx_spulse_offset[0] = (rx_bitslip_cnt + rx_dlpulse_alignment*33)*UI14
rx_spulse_offset_sign[0] = 1’b0;
    2. Detect rollover of asynchronous pulse time:

      The rx_apulse_time[pl] signal represents an asynchronous pulse time of each physical lane in a 28-bit format, where bit [27:16] represent asynchronous pulse time in nanoseconds (ns) and bit [15:0] represent asynchronous pulse time in fractional nanoseconds (fns).

      Two types of rollover are possible:
      1. Natural rollover from bit 27 to bit 28 when the value reaches 28'hFFF_FFFF. Before rollover, bit [27:24] is 4'hF.
      2. Billion rollover when the TOD reaches one billion ns or 48'h3B9A_CA00_0000 in hex value. Before rollover, bit [27:24] is 4'h9. 
      Given rx_apulse_time_max is largest rx_apulse_time from all physical lanes,
for (pl = 0; pl < PL; pl++){
  if (rx_apulse_time_max - rx_apulse_time[pl] > 29'h01F4_0000){
    if (rx_apulse_time_max[27:24] == 4'hF) {
      rx_apulse_time[pl] = rx_apulse_time[pl] + 29'h1000_0000
    } else {
      rx_apulse_time[pl] = rx_apulse_time[pl] + 29'h0A00_0000
    }
  }
}
    3. Calculate the actual time of RX Alignment Marker (AM) at RX PMA parallel data interface:
      // fl to pl map
pl = fl to pl map(f) = (fl-(fl%pl_fl_map))/pl_fl_map
      // vl to pl map
<See PL in RX Virtual Lane Offset Calculation for No FEC Variants>
      • FEC variants: 
        for (fl = 0; fl < FL; fl++) {
local_pl = fl_to_pl_map(fl)
rx_am_actual_time[fl]= (rx_apulse_time[local_pl])
   + (rx_apulse_offset_sign[local_pl] ? 
      –rx_apulse_offset[local_pl] : rx_apulse_offset[local_pl])
      – (rx_apulse_wdelay[local_pl])
   + (rx_spulse_offset_sign[fl] ? 
      -rx_spulse_offset[fl] : rx_spulse_offset[fl]))
}
      • No FEC variants: 
        for (vl = 0; vl < VL; vl++) {
local_pl = vl_to_pl_map(vl)
rx_am_actual_time[vl] = (rx_apulse_time[local_pl]) 
   + (rx_apulse_offset_sign[local_pl] ? 
      –rx_apulse_offset[local_pl] : rx_apulse_offset[local_pl])
    – (rx_apulse_wdelay[local_pl])
   + (rx_spulse_offset_sign[vl] ? 
      -rx_spulse_offset[vl] : rx_spulse_offset[vl])
}
    4. Determine RX reference lane:
      • FEC variants:
        rx_am_actual_time[rx_ref_fl] is maximum value out of rx_am_actual_time[FL-1:0]
rx_ref_pl = fl_to_pl_map(rx_ref_fl)

where rx_am_actual_time[fl] is max(rx_am_actual_time[FL-1:0])
      • No FEC variants:
        rx_am_actual_time[rx_ref_vl] is maximum value out of rx_am_actual_time[VL-1:0]
rx_ref_pl = vl_to_pl_map(rx_ref_vl)

where rx_am_actual_time[vl] is max(rx_am_actual_time[VL-1:0])
  6. Calculate RX offsets:
    1. Calculate RX TAM adjust:
      • FEC variants: 
        rx_tam_adjust_sim = 
   (rx_const_delay_sign ? –rx_const_delay : rx_const_delay) 
 + (rx_apulse_offset_sign[rx_ref_pl] ? 
     –rx_apulse_offset[rx_ref_pl] : rx_apulse_offset[rx_ref_pl]
 – (rx_apulse_wdelay[rx_ref_pl])
 + (rx_spulse_offset_sign[rx_ref_fl] ? 
     -rx_spulse_offset[rx_ref_fl] : rx_spulse_offset[rx_ref_fl])
      • No FEC variants: 
        rx_tam_adjust_sim = 
   (rx_const_delay_sign ? –rx_const_delay : rx_const_delay) 
 + (rx_apulse_offset_sign[rx_ref_pl] ? 
     –rx_apulse_offset[rx_ref_pl] : rx_apulse_offset[rx_ref_pl])
 – (rx_apulse_wdelay[rx_ref_pl])
 + (rx_spulse_offset_sign[rx_ref_vl] ? 
    -rx_spulse_offset[rx_ref_vl] : rx_spulse_offset[rx_ref_vl])
      For hardware run with PTP Timestamp accuracy mode set to Advanced: 
      rx_tam_adjust = (rx_tam_adjust_sim) 
 + (rx_routing_adj_sign[rx_ref_pl] ? – rx_routing_adj[rx_ref_pl] 
                                     : rx_routing_adj[rx_ref_pl])
      For routing delay adjustment information, refer to Routing Delay Adjustment for Advanced Timestamp Accuracy Mode.
      For all other cases: 
      rx_tam_adjust = rx_tam_adjust_sim

      Convert TAM adjust to a 32-bit 2's complement number: 

      rx_tam_adjust_2c = rx_tam_adjust
where rx_tam_adjust is a 32-bit 2's complement number
    2. Calculate RX extra latency:
      Convert unit of RX PMA delay from UI to nanoseconds: 
      rx_pma_delay_ns = rx_pma_delay_ui * UI14
      RX extra latency is a negative adjustment. To indicate the negative adjustment, set the most-significant register bit to 1. Total up all extra latency together: 
      rx_extra_latency[31] = 1
rx_extra_latency[30:0] = rx_pma_delay_ns + rx_external_phy_delay
    3. Calculate RX virtual lane offsets:
      Attention: This step is not applicable for 10G and 25G Ethernet data rates. You must skip this step for these rates.

      Using determined reference virtual lane, assign RX virtual lane offset values as described in Virtual Lane Order and Offset Values.

      • For FEC KP-FEC or FEC LL-FEC variants:
        for (vl = 0; vl < VL; vl++) {
   rx_vl_offset[vl] = [vl - (vl % PL)] / PL * 68 * UI14
}
      • For FEC KR-FEC variants:
        for (vl = 0; vl < VL; vl++) {
   rx_vl_offset[vl] = [vl - (vl % PL)] / PL * 66 * UI14
}
      • For no FEC 100G Ethernet rate variants:
        for (vl = 0; vl < VL; vl++) {
   rx_vl_offset[vl] = 2 * UI14
}
      • For no FEC 50G Ethernet rate variants:
        for (vl = 0; vl < VL; vl++) {
   rx_vl_offset[vl] = 0.5 * UI14
}
  7. Write the determined RX reference lane into IP:
    Attention: This step is not applicable for 10G and 25G Ethernet data rates. You must skip this step for the specified rates. 
    csr_write(ptp_ref_lane.rx_ref_lane, rx_ref_pl)
  8. Write the calculated RX offsets to IP:
    1. Write RX virtual lane offsets:
      Attention: This step is not applicable for 10G and 25G Ethernet data rates. You must skip this step for the specified rates. 
      for (vl = 0; vl < VL; vl++) {
   csr_write(rx_ptp_vl_offset_<vl>, rx_vl_offset[vl])
}
    2. Write RX extra latency:
      csr_write(rx_ptp_extra_latency, rx_extra_latency)
    3. Write RX TAM adjust:
      csr_write(ptp_rx_tam_adjust, rx_tam_adjust_2c)
  9. Continue UI value measurement. Follow steps 1 through 7 mentioned in section RX UI Adjustment.
    For simulation or hardware run with 0 ppm setup, you can skip the measurement and program 0 ppm UI value defined in .
  10. Notify soft PTP that uses flow configuration is completed. 
    csr_write(ptp_rx_user_cfg_status.rx_user_cfg_done, 1'b1)
  11. Wait until RX PTP is ready.
    You can monitor the status via one of the following:
    • Output port:
      o_rx_ptp_ready = 1'b1
    • Polling via CSR:
      csr_read(ptp_status.rx_ptp_ready) = 1’b1
  12. RX PTP is up and running.
    1. Adjust RX UI value.

      Perform the RX UI adjustment occasionally to prevent time counter drift from golden time-of-day in the system. Follow steps 1 through 8 described in RX UI Adjustment.

      Note: UI measurement is a long process in simulation. Therefore, for simulation, Intel recommends skipping this step and program a 0 ppm value. For more details, refer to UI Value and PMA Delay.
14 The UI format differs from the format of other variables. UI uses the {4-bit ns, 28-bit fractional ns} format. Other variables defined in this flow use the {N-bit ns, 16-bit fractional ns} format, where N is the largest number to store the calculation's max value. If you use UI format in your calculation, you must convert your result to a 16-bit fractional ns format.
Level Two Title