Hard Processor System Technical Reference Manual: Agilex™ 5 SoCs

ID 814346
Date 4/01/2024
Public
Document Table of Contents

5.1.6.10.1. Enhancements to Scheduled Traffic

The IEEE 802.1Qbv-2015 standard defines the schedule for each of the queues on every egress port which makes the implementation aware of traffic arrival schedule. This information can be used to block the lower priority traffic from transmission in this time window/slot. This ensures that scheduled traffic is forwarded from sender to receiver through all the network nodes with a deterministic delay.
Figure 78. Time Aware Shaper Implementation for Traffic Scheduling

One of the important requirements to achieve a low latency is to ensure there are no interfering frames during the scheduled windows that are reserved for high priority traffic. The use of scheduled traffic imposes limitations while starting a transmission.

As shown in the following figure, if an interfering frame begins transmission just before the start of a reserved time period, it can extend transmission into the reserved window, and potentially interfere with higher priority traffic. Therefore, a guard band whose size is equal to the largest possible interfering frame is required before the window starts.
Figure 79. Implementing a Guard Band to Avoid Delays Due to Interfering Frames

A larger guard band equates to a less efficient use of network bandwidth. However, with the implementation of IEEE802.1Qbu (frame preemption), this issue is addressed. Frame preemption breaks the interfering frame into smaller fragments. Therefore, the guard band needs to be only as large as the largest possible interfering fragment instead of the largest possible interfering frame.

During the guard band, only the frames that can complete the transmission of the entire frame before the next gate close event are permitted. This ensures that the high priority traffic can always start at the beginning of the window reserved for it.

The following operations are required to implement EST:
  • Implementation of gate control list (GCL)
  • Enforcing gate controls in the scheduler
  • Accounting for gate open (O) duty cycle in the computation of idleSlope (CBS)
Note:
Common EST support definitions:
  • Gate control list - The list in the hardware memory that holds the gate controls and the associated time intervals.
  • Gate controls - For a given schedule (row in gate control list) there is a gate open (O) and gate close (C) state associated with each traffic class (TC). The set of O or C values, whose width is same as configured TCs is called the gate controls. As an example, CCOCOOCO means TC7=C, TC6=C, TC5=O and so on.
  • Time interval - Time interval (in nano seconds) is a 16, 20, or 24-bit configurable field in the gate control list that indicates the time for which the associated gate controls are valid.
  • Base time register - Each gate control list is associated with a 64-bit base time register that holds the start time (in PTP format) for the list.

Implementation of Gate Control List

The gate control list (GCL) has the following two parts:
  • Time interval - Defines the time in nanoseconds for which the gate controls are valid and must be applied before reading the next gate controls from the list. Configurable width of 16, 20 or 24 to represent a maximum of 64 us, 1 ms, or 16 ms schedule interval respectively. Supports left shift of the time interval up to 7 bits to be able to apply a multiplication factor from 1 to 128 ns (in steps of 2^n). The maximum value (post shifting) of this field must be 999,999,999 ns. The time interval (TI) value must be large enough to account for the implementation delays. Any value less than that cannot be accurately implemented. To account for the implementation constraints, the minimum time interval for CGL entry must be equal to the time required to transmit fragment_size + IPG/EIPG + preamble overheads.
  • Gate control - Defines the open (O) represented by logic 1 or close (C) represented by logic 0 state for the gate of each TC.
The following figure shows a block diagram from the IEEE 802.1Qbv specification that illustrates how the gate control list governs the gate close (C) and open (O) events based on the schedule provided for each event.
Figure 80. Block Diagram from IEEE 802.1 Qbv Specification - GCL Governing Gate Close and Open Events
The implementation of GCL consists of the following two gate control lists:
  • HWOL - Hardware owned list which is a list for hardware access.
  • SWOL - Software owned list which is a list for software access.
The access to these lists is mutually exclusive. The ownership of the list is set by hardware in the SWOL field of MTL_EST_Status register. The following table provides the implementation details of the GCL.
Table 159.  External Memory Used for Holding the Two Gate-Control Lists
Gate Control (up to 8 bits) Time Interval (ns) 16, 20, or 24 bits Type
OOCCCCCC T0 HWOL or SWOL
OOOOCCCC T1
CCCCOOCC T2
CCCCCCOO T3
I I
I I
OCOCOCOO  
OOOOCCCC T last
OOOOCCCC T0 SWOL or HWOL
OOOOCCCC T1
OOCCCCCC T2
OOOOCCCC T3
I I
I I
OCOCCCCC  
OOCCCCCC T last

Gate Control List Registers

The specification implements the gate control list registers through indirect addressing using the GCL_Control and GCL_Data registers as shown in the following list and described in the table:

  1. 64-bit Base time register (BTR)
  2. 40-bit Cycle time register(CTR)
  3. m-bit Time extension register (TER)
  4. n-bit List length register (LLR) (n depends of the configured GCL depth)
Table 160.  Gate Control List Registers
Registers Name Description

Base Time Register (BTR)

Base time register is a 64-bit register that holds the start time to execute the GCL. The format of the BTR is same as the PTP format (upper 32-bits holds time in seconds and lower 32 bits hold time in nano seconds).

Once the execution of a given list begins, the implementation can update the value in BTR to indicate the next list execution begin time.

Cycle Time Register (CTR)

Cycle time register is a 40-bit register that holds the time at which the execution of the GCL must be repeated. The cycle time register consists of an 8-bit value in seconds, and a 32-bit value in nanoseconds (similar to the PTP time format with truncated seconds register).

For a given gate control list the start time is Base Time + N * Cycle Time where N is an integer value representing the iteration number starting with 0 for the first iteration. If the gate control list execution takes longer than the cycle time, then the list is truncated at the cycle time and the subsequent loop begins at cycle time.

Time Extension Register (TER) Time extension register is a m-bit (where m = configured time interval width + 7) register that holds the amount of time (in nano seconds) the current gate control list can be extended before switching to the new gate control list. This avoids smaller fragments of the current list being executed before switching to a new list.

List Length Register (LLR)

List length register is an n-bit register (when n is 7, 8, 9, 10, or 11 for a GCL configured depth of 64, 128, 256, 512, or 1024 respectively) that holds the integer value of the length of the GCL (that is the number of valid rows in each GCL). The processing of the GCL stops after the number of rows read equals to the LLR value.

Transmission Gating Implementation

A bridge or an end station can be enhanced to allow transmission from each TC that is yet to be scheduled relative to a known timescale. To achieve this, a transmission gate is associated with each TC; the state of the transmission gate determines if the queued frames can be selected for transmission.

For a TC, the transmission gate can be in one of the following two states:
  1. OPEN - queued frames are selected for transmission, in accordance with the definition of the transmission selection algorithm associated with the TC.
  2. CLOSED - queued frames are not selected for transmission.

A frame on a traffic class queue is not available for transmission if the transmission gate is in the closed state or if there is insufficient time available to transmit the entirety of that frame before the next gate-close event associated with that queue.

The implementation has visibility to the current schedule of gate controls and the immediate next schedule of the gate controls. So the maximum gate open period does not exceed the sum of the two time intervals. This is because, a frame is selected for transmission only if the gate is currently open and the duration of gate open interval is greater than the time taken to transfer the entire frame.

The implementation must know the frame size before the transmission, so that the transmission overruns can be avoided and only the frames that can complete are scheduled at all times. The implementation adequately compensates for the implementation delays in the data transfer from the buffer to the line by offsetting the current time with all the relevant delays (provided by the CTOV field of MTL_EST_Control register). This ensures that the schedule provided is always accurately implemented at the line.
Note:
  • A 20 byte overhead is added to the packet size to account for the IFG and preamble overheads on the line. Therefore for a 128 byte frame to be transmitted, the gate open window must at the least be able to accommodate 148 bytes.
  • In 100 Mbps mode of operation, the rounding down error is about 0.4%. For example, the gate open duration must be at least 1024 bytes for transmitting 1000 byte frames at a speed of 100 Mbps (20 bytes for line overheads, and 4 bytes for rounding down error)

Idle Slope Computation Updates

When EST is enabled, credit is accumulated only when the gate is open; therefore, the effective data rate of the idleSlope must be increased to reflect the duty cycle for the transmission gate associated with the queue.

The idleSlope is computed based on the GateOpenTime and OperCycleTime values. Program the idleSlope registers (implemented one per CBS enabled TC) based on the following equation. The existing MTL register has sufficient bit width to accommodate the new values for idleSlope.

idleSlope = (operIdleSlope(N) * OperCycle/GateOpenTime)

Installing a New GCL

When a new software programmed GCL is available and must be executed at the new BTR value, the switch to the new GCL can happen in one of the following ways:

  • New Base Time Aligned with Current Schedule - If the choice of cycle time for the new gating cycle is unchanged from the cycle time for the current gating cycle, and if the BTR chosen for the new gating cycle (new BTR) is an integer multiple of the current cycle time (+ current BTR), then the new gating cycle exactly lines up with the old gating cycle, that is, the cycle start times for the new gating cycle is same as they would have been for the old configuration. This could be considered to be the ideal case and allows the new gating cycle to be installed and executed with no timing issues. The implementation completes the execution of an iteration of the current list and switches to the new list at the beginning of the BaseTime listed in the new list.
  • New Base Time Unaligned with Current Schedule - If new CycleTime differs from current CycleTime or new BaseTime is in the future and is not an integer multiple of current CycleTime, then the old and new cycles do not line up; when new BaseTime is reached (when the new configuration is installed and starts to execute), the last old cycle is normally truncated to start the first new cycle. This could be undesirable if it results in a very short last old cycle; arguably it would be better to simply extend the penultimate old cycle by that small amount, rather than starting a very short cycle. The cycle time extension register (related to the current config list) allows this extension of the last old cycle to be done in a defined way; if the last complete old cycle ends normally in less than current cycle time extension (TER) ns before the new base time, then the last complete cycle before new BaseTime is reached is extended so that it ends at new BaseTime.

Impact of Transmit Scheduling Algorithms on EST

When you use EST in isolation, the gate control list manages the final open/close state of the transmit class along with the checks carried out by the transmission selection algorithm in MTL. As the gate controls operate on a predefined repetitive schedule, it is recommended to use strict priority or credit based shaper (CBS) scheduling algorithms.

Other algorithms such as the weighted round robin (WRR), deficit weighted round robin (DWRR), and weighted fair queuing (WFQ) implement masking of the transmit channels based on the current winning transmit channel. The algorithm is applied only among the group of transmit channels that open simultaneously. To ensure transmit channels whose gates are open get priority, these algorithms are modified to treat gate open transmit channels and gate closed transmit channels as separate groups giving priority to gate open transmit channels.

For example, consider 4 transmit channels (TC3, TC2, TC1, TC0) with weights 4:3:2:1; TC3 & TC2 are in open state in one slot, while TC1 & TC0 are in open state in another slot. In this case, the scheduler works as follows:
  1. In the first slot, the TC3 & TC2 are serviced in the ratio of 4:3 for the duration the slot is open.
  2. In the second slot, the transmit channels TC1 & TC0 are serviced in the ratio of 2:1.
  3. Fresh arbitration is started every time a slot is opened.

In other words, the traffic does not get distributed in the intended ratio of 4:3:2:1; but as two groups with different ratios and only for the duration of the slot when the gates are open continuously.

When multiple queues map to one TC, EMAC has a first-stage scheduler (round robin) for the queue to TC selection. When EST is enabled, EMAC uses an enhanced round robin. Based on the open window size of EST, the scheduler masks the round robin for queues whose frames do not fit in the window size. This ensures that the scheduler picks the correct queue in the round robin, considering the frame sizes too.

EST Errors

EST can have the following errors:
  • BTR programming error - This error indicates that the value of the new BTR is less than the current time. EMAC attempts to recover from this error by recursively adding the new cycle time to the new BTR.
  • Constant gate control error - This error occurs when the list length register (LLR) is 1 and the programmed time interval (TI) value after the optional left shifting is more than or equal to the cycle time register (CTR). This implies that the gates are either always closed or always open based on the gate control values. As this is not supported, it is reported as an error.