8. System Interconnect

Intel® Arria® 10 Hard Processor System Technical Reference Manual

Download PDF

ID 683711

Date 1/10/2023

Version

Public

A newer version of this document is available. Customers should click here to go to the newest version.

Visible to Intel only — Ixiasoft

View Details

8. System Interconnect

The components of the hard processor system (HPS) communicate with one another, and with other portions of the SoC device, through the system interconnect. The system interconnect consists of the following blocks:

The main level 3 (L3) interconnect
The SDRAM L3 interconnect
The level 4 (L4) buses

The system interconnect is the main communication bus for the MPU and all IPs in the SoC device.

The system interconnect is implemented by the Arteris® FlexNoC™ network-on-chip (NoC) interconnect module.

The system interconnect supports the following features:

An L3 interconnect that provides high-bandwidth routing between masters and slaves in the HPS
SDRAM L3 interconnect, providing access to a hard memory controller in the FPGA fabric through a multiport front end (MPFE) scheduler for sharing the external SDRAM between multiple masters in the SoC device
Independent L4 buses running in several clock domains. These buses handle data traffic for low- to mid-level bandwidth slave peripherals and accesses to peripheral control and status registers throughout the address map.
On-chip debugging and tracing capabilities
Security firewalls with the following capabilities:
- Secure or nonsecure access configured per peripheral
- Privileged or user access configured per peripheral (for some peripherals)
- Security optionally configured per transaction at the master
Quality of service (QoS) with three programmable levels of service on a per-master basis

About the System Interconnect

Features of the System Interconnect

The system interconnect supports high-throughput peripheral devices. The system interconnect has the following characteristics:

Byte oriented address handling
Data bus width up to 128 bits
Arm* TrustZone* -compliant firewall and security support, with additional security features configurable per master
Programmable quality-of-service (QoS) optimization
Dedicated SDRAM L3 interconnect, providing access and scheduling for the hard memory controller in the FPGA portion of the SoC device
On-chip debug and tracing capabilities
Multiple independent L4 buses with independent clock domains and protocols
The L4 buses are each connected to a master in the main L3 interconnect for control and status register access. Each L4 bus is 32 bits wide and is connected to multiple slaves. The L4 buses also provide a low- to mid-level tier of the system interconnect for lower-bandwidth slave peripherals. Each L4 bus is 32 bits wide and operates in a separate clock domain.

System Interconnect Block Diagram and System Integration

System Interconnect Block Diagram

The following figures show the system interconnect, including the main L3 interconnect, SDRAM L3 interconnect, and L4 buses.

Figure 27. High-Level System Interconnect Block DiagramThe following figure shows the relationships among the system interconnect and other major SoC components.

Connectivity

Arria 10 HPS Master-to-Slave Connectivity Matrix

The system interconnect is a highly efficient packet-switch network.

Each system interconnect packet carries a transaction between a master and a slave. The following figure shows the connectivity of all the master and slave interfaces in the system interconnect.

Figure 28. Master-to-Slave Connectivity

On-Chip RAM

Figure 29. On-Chip RAM Connections HPS masters connecting to the on-chip RAM

Peripherals Connections

Figure 30. Peripherals Connections HPS masters connecting to HPS peripherals

System Connections

Figure 31. System Connections HPS masters connecting to HPS system peripherals

HPS-to-FPGA Bridge

Figure 32. HPS-to-FPGA Bridge Connections HPS masters connecting to the HPS-to-FPGA bridges

SDRAM Connections

Figure 33. SDRAM ConnectionsHPS masters connecting to the SDRAM scheduler. Note that cache misses to the MPU Accelerator Coherency Port (ACP) pass through a secondary firewall.

System Interconnect Architecture

The system interconnect has a transaction-based architecture that functions as a partially-connected fabric. Not all masters can access all slaves.

The system interconnect is a packet-oriented network on chip (NoC), in which each packet carries a transaction between a master and a slave. The interconnect provides interface widths up to 128 bits, connecting to the L4 slave buses and to HPS and FPGA masters and slaves.

The system interconnect provides low-latency connectivity to the following bridges:

HPS-to-FPGA bridge
Lightweight HPS-to-FPGA bridge
FPGA-to-HPS bridge
Three FPGA-to-SDRAM ports

SDRAM L3 Interconnect Architecture

The SDRAM L3 interconnect provides access to the hard memory controller in the FPGA portion of the SoC device.

The SDRAM L3 interconnect is part of the system interconnect, and includes these components:

SDRAM scheduler
SDRAM adapter

The SDRAM L3 interconnect operates in a clock domain that is 1/2 the speed of the hard memory controller clock.

Arria 10 HPS Secure Firewalls

You can use the system interconnect firewalls to enforce security policies for slave and memory regions in the system interconnect.

The firewalls support the following features:

Secure or non-secure access configurable per peripheral
Privileged or user access configurable for some peripherals
Per-transaction security

Each firewall contains the security configuration registers (SCRs) that set security policies and define which transactions are allowed to go through the firewall. If you want to generate an error any time a transaction is blocked in the firewall, then you must set the error_response bit in the global register of the noc_fw_ddr_l3_ddr_scr module at base address 0xFFD13400. If this bit is clear, transactions blocked by the firewall return random data.

Note: Future devices may not support the return of random data and may only support an error response for blocked firewall transactions. For designs that may be ported to future devices, Intel recommends you to set the error_response bit in the global register.

There are five main firewalls in the HPS:

Peripheral
System
HPS-to-FPGA
On-Chip RAM
SDRAM (which includes DDR and DDR L3 firewalls)

About the Rate Adapters

The L3 interconnect contains a rate adapter wherever a low-bandwidth channel transfers data to a high-bandwidth channel.

About the SDRAM L3 Interconnect

The hard processor system (HPS) provides a specialized SDRAM L3 Interconnect dedicated to SDRAM accesses.

The SDRAM L3 Interconnect contains an SDRAM scheduler and an SDRAM adapter. The SDRAM scheduler functions as a multi-port front end (MPFE) for multiple HPS masters. The SDRAM adapter is responsible for connecting the HPS to the SDRAM hard memory controller in the FPGA portion of the device. The SDRAM L3 interconnect is part of the system interconnect.

Features of the SDRAM L3 Interconnect

The SDRAM L3 interconnect supports the following features:

Connectivity to the SDRAM hard memory controller supporting:
- DDR4-SDRAM
- DDR3-SDRAM
Integrated SDRAM scheduler, functioning as a multi-port front end (MPFE)
Configurable SDRAM device data widths
- 16-bit, with or without 8-bit error-correcting code (ECC)
- 32-bit, with or without 8-bit ECC
- 64-bit, with or without 8-bit ECC
High-performance ports
- MPU subsystem port
- Main L3 interconnect port
- Three FPGA ports
  - Two 32-, 64-, or 128-bit ports
  - One 32- or 64-bit port
- Firewall and security support port

Note: At system startup, the SDRAM I/O pins can be configured separately from the FPGA fabric, allowing the SoC HPS to boot before any soft logic is configured in the FPGA.

SDRAM L3 Interconnect Block Diagram and System Integration

The SDRAM L3 interconnect is composed of two main blocks: SDRAM adapter and SDRAM scheduler.

The SDRAM adapter is responsible for bridging the hard memory controller in the FPGA fabric to the SDRAM scheduler. The adapter is also responsible for ECC generation and checking.

The ECC register interface provides control to perform memory and ECC logic diagnostics.

The SDRAM scheduler is an MPFE, responsible for arbitrating collisions and optimizing performance in traffic to the SDRAM controller in the FPGA portion of the device.

Three Arm* Advanced Microcontroller Bus Architecture ( AMBA* ) Advanced eXtensible Interface ( AXI* ) ports are exposed to the FPGA fabric, allowing soft logic masters to access the SDRAM controller through the same scheduler unit as the MPU subsystem and other masters within the HPS. The MPU has access to the SDRAM adapter's control interface to the hard memory controller.

Figure 34. SDRAM L3 Interconnect Block Diagram

The hard memory controller in the FPGA portion of the device has a dedicated connection to the SDRAM L3 interconnect. This connection allows the hard memory controller to become operational before the rest of the FPGA has been configured.

About the SDRAM Scheduler

The SDRAM scheduler functions as a multi-port front end (MPFE), scheduling transactions from multiple masters to the SDRAM.

The SDRAM scheduler supports the following masters:

The MPU
The L3 system interconnect
The FPGA-to-SDRAM bridges

The SDRAM scheduler arbitrates among transactions initiated by the masters, and determines the order of operations. The scheduler arbitrates among the masters, ensuring optimal interconnect performance based on configurable quality-of-service settings.

The SDRAM scheduler can be configured through the registers.

About Arbitration and Quality of Service

When multiple transactions need the same interconnect resource at the same time, arbitration logic resolves the contention. The quality-of-service (QoS) logic gives you control over how the contention is resolved.

Arbitration and QoS logic work together to enable optimal performance in your system. For example, by setting QoS parameters, you can prevent one master from using up the interconnect's bandwidth at the expense of other masters.

The system interconnect supports QoS optimization through programmable QoS generators. The QoS generators are located on interconnect initiators, which correspond to master interfaces. The initiators insert packets into the interconnect, with each packet carrying a transaction between a master and a slave. Each QoS generator creates control signals that prioritize the handling of individual transactions to meet performance requirements.

Arbitration and QoS in the HPS system interconnect are based on the following concepts:

Priority—Each packet has a priority value. The arbitration logic generally gives resources to packets with higher priorities.
Urgency—Each master has an urgency value. When it initiates a packet, it assigns a priority equal to its urgency.
Pressure—Each data path has a pressure value. If the pressure is raised, packets on that path are treated as if their priority was also raised.
Hurry—Each master has a hurry value. If the hurry is raised, all packets from that master are treated as if their priority was also raised.

Proper QoS settings depend on your performance requirements for each component and peripheral, and for system performance as a whole. Intel recommends that you become familiar with QoS optimization techniques before you try to change the QoS settings in the HPS system interconnect.

About the Service Network

The service network is physically separate from the NoC datapath.

Through the service network, you can perform these tasks:

Access internal interconnect registers
Update master and slave peripheral security features

About the Observation Network

The observation network connects probes to the CoreSight* trace funnel through the debug channel. It is physically separate from the NoC datapath.

The observation network connects probes to the observer, which is a port in the CoreSight* trace funnel. Through the observation network, you can perform these tasks:

Enable error logging
Selectively trace transactions in the system interconnect
Collect HPS transaction statistics and profiling data

The observation network consists of probes in key locations in the interconnect, plus connections to observers. The observation network works across multiple clock domains, and implements its own clock crossing and pipelining where needed.

The observation network sends probe data to the CoreSight subsystem through the ATB interface. Software can enable probes and retrieve probe data through the interconnect observation registers.

Functional Description of the System Interconnect

The system interconnect provides access to a 4 GB address space.

Address spaces are divided into one or more nonoverlapping contiguous regions.

Figure 35. HPS Address Space RelationshipsPositions of HPS address spaces. For readability, some memory regions are shown out-of-scale to their addresses.

The window regions provide access to other address spaces. The thin black arrows indicate which address space is accessed by a window region (arrows point to accessed address space).

The following table shows the base address and size of each region that is common to the L3 and MPU address spaces.

Table 44. Common Address Space Regions
Region Name	Description	Base Address	Size
FPGASLAVES	FPGA slaves	0xC0000000	960 MB
PERIPH	Peripheral	0xFC000000	64 MB
LWFPGASLAVES ⁸	Lightweight FPGA slaves	0xFF200000	2 MB

System Interconnect Address Spaces

The system interconnect supports multiple address spaces.

Each address space uses some or all of the 4 GB address range. The address spaces overlap. Depending on the configuration, different address spaces are visible in different regions for each master.

The following address spaces are available:

The L3 address space
The MPU address space
The SDRAM address space

Arria 10 HPS Available Address Maps

The following figure shows the default system interconnect address maps for all masters. The figure is not to scale.

Figure 36. Address Maps for System Interconnect Masters

Notes on Address Maps

⁽¹⁾ Transactions on these addresses are directly decoded by the SCU and L2 cache.

⁽²⁾ The MPU accesses SDRAM through a dedicated port to the SDRAM scheduler. The SDRAM window size also depends on L2 cache filtering.

⁽³⁾ This region can be configured to access on-chip RAM, by using the noc_addr_remap_value, noc_addr_remap_set, and noc_addr_remap_clear registers, in the system manager.

⁽⁴⁾ The following peripherals can master the interconnect:

Ethernet MACs
USB-2.0 OTG controllers
NAND controller
ETR
SD/MMC controller

⁴To access registers that are connected to the HPS-to-FPGA AXI* master bridge, you need to set the base address of your slave interface starting from 0xC0000000. The HPS-to-FPGA AXI* master bridge can be connected to any type of slave interface such as APB* and Avalon® .

For the MPU L3 master, either the boot ROM or on-chip RAM can map to address 0x0 and obscure the lowest 128 KB or 256 KB of SDRAM.

At boot time, the MPU does not have access to the SDRAM address space from the top of ROM or on-chip RAM to 0x00100000. This is because the MPU's SDRAM access is controlled by the MPU L2 filter registers, which only have a granularity of 1 MB. After booting completes, the MPU can change address filtering to use the lowest 1 MB of SDRAM.

For non-MPU masters, either the on-chip RAM or the SDRAM maps to address 0x0. When mapped to address 0x0, the on-chip RAM obscures the lowest 256 KB of SDRAM for non-MPU masters.

Arria 10 HPS L3 Address Space

The Arria 10 HPS L3 address space is 4 GB and applies to all L3 masters except the MPU subsystem.

The system interconnect noc_addr_remap_value, noc_addr_remap_set, and noc_addr_remap_clear registers, in the System Manager, determine if the address space starting at address 0x0 is mapped to the on-chip RAM (256 KB) or the SDRAM. SDRAM is mapped to address 0x0 on reset.

The L3 address space configurations contain the regions shown in the following table:

Table 45. L3 Address Space Regions
Description	Condition	Base Address	End Address	Size
SDRAM window (without on-chip RAM)	`remap0` set, `remap1` clear ⁹	0x00000000	0xBFFFFFFF	3 GB
On-chip RAM (low mapping)	`remap0` set, `remap1` set ⁹	0x00000000	0x0003FFFF	256 KB
SDRAM window (with on-chip RAM)	`remap0` set, `remap1` set ⁹	0x00040000	0xBFFFFFFF	3145471 KB = 3 GB - 256 KB
HPS-to-FPGA	Not visible to FPGA-to-HPS bridge.	0xC0000000	0xFBFFFFFF	960 MB
System trace macrocell	Always visible to DMA and FPGA-to-HPS	0xFC000000	0xFEFFFFFF	48 KB
Debug access port	Not visible to master peripherals. Always visible to other masters.	0xFF000000	0xFF1FFFFF	2 MB
Lightweight HPS-to-FPGA	Not visible to master peripherals. Always visible to other masters.	0xFF200000	0xFF3FFFFF	2 MB
Peripherals	Not visible to master peripherals. Always visible to other masters.	0xFF800000	0xFFFDFFFF	8064 KB
On-chip RAM (high mapping)	Always visible	0xFFE00000	0xFFFE3FFF	256 KB

The boot ROM and internal MPU registers (SCU and L2) are not accessible to L3 masters.

Cache coherent memory accesses have the same view of memory as the MPU.

SDRAM Window Region

The SDRAM window region is 3 GB and provides access to the bottom 3 GB of the SDRAM address space.

On-Chip RAM Region, Low Mapping

The system interconnect noc_addr_remap_value register determines if the 256 KB starting at address 0x0 is mapped to the on-chip RAM or the SDRAM. The SDRAM is mapped to address 0x0 on reset.

HPS-to-FPGA Slaves Region

The HPS-to-FPGA slaves region provides access to 960 MB of slaves in the FPGA fabric through the HPS-to-FPGA bridge.

Lightweight HPS-to-FPGA Slaves Region

The lightweight HPS-to-FPGA slaves provide access to slaves in the FPGA fabric through the lightweight HPS-to-FPGA bridge.

Peripherals Region

The peripherals region includes slaves connected to the L3 interconnect and L4 buses.

On-Chip RAM Region, High Mapping

The on-chip RAM is always mapped (independent of the boot region contents).

MPU Address Space

The MPU address space is 4 GB and applies to MPU masters.

Addresses generated by the MPU are decoded in three ways:

By default, MPU accesses to locations between 0x100000 (1 MB) to 0xC0000000 (3 GB) are made to the SDRAM controller.
Addresses in the SCU and L2 register region (0xFFFFC000 to 0xFFFFFFFF) are the SCU and L2 bus.
Accesses to all other locations are made to the L3 interconnect.

The MPU L2 cache controller contains a master connected to the main L3 interconnect and a master connected to the SDRAM L3 interconnect.

The MPU address space contains the following regions:

Table 46. MPU Default Address Space Regions
Description	Condition	Base Address	End Address	Size
Boot ROM	`remap0` clear, `remap1` clear ¹⁰	0x00000000	0x0001FFFF	128 KB
SDRAM window	Always visible	0x00100000	0xBFFFFFFF	3071 MB (3 GB – 1 MB)
HPS-to-FPGA	Always visible	0xC0000000	0xFBFFFFFF	960 MB (1 GB – 64 MB)
System trace macrocell	Always visible	0xFC000000	0xFFEFFFFF	48 KB
Debug access port	Always visible	0xFF000000	0xFF1FFFFF	2 MB
Lightweight HPS-to-FPGA	Always visible	0xFF200000	0xFF3FFFFF	2 MB
Peripherals	Always visible	0xFF800000	0xFFDFFFFF	6 MB
On-chip RAM	Always visible	0xFFE00000	0xFFFE3FFF	256 KB
Boot ROM	Always visible	0xFFFC0000	0xFFFDFFFF	128 KB
SCU and L2 registers	Always visible	0xFFFFC000	0xFFFFFFFF	16 KB

Boot Region

The boot region is 1 MB, based at address 0x0. The boot region is visible to the MPU only when the L2 address filter start register is set to 0x100000. The L3 noc_addr_remap_value control register determines if the boot region is mapped to the on-chip RAM or the boot ROM.

The boot region is mapped to the boot ROM on reset. Only the lowest 128 KB of the boot region are legal addresses because the boot ROM is only 128 KB.

When the L2 address filter start register is set to 0, SDRAM obscures access to the boot region. This technique can be used to gain access to the lowest SDRAM addresses after booting completes.

SDRAM Window Region

The SDRAM window region provides access to a large, configurable portion of the 4 GB SDRAM address space. The address filtering start and end registers in the L2 cache controller define the SDRAM window boundaries.

The boundaries are megabyte-aligned. Addresses within the boundaries route to the SDRAM master, while addresses outside the boundaries route to the system interconnect master.

Addresses in the SDRAM window match addresses in the SDRAM address space. Thus, the lowest 1 MB of the SDRAM is not visible to the MPU unless the L2 address filter start register is set to 0.

HPS-to-FPGA Slaves Region

The HPS-to-FPGA slaves region provides access to slaves in the FPGA fabric through the HPS-to-FPGA bridge. Software can move the top of the SDRAM window by writing to the L2 address filter end register. If higher addresses are made available in the SDRAM window, part of the FPGA slaves region might be inaccessible to the MPU.

Lightweight HPS-to-FPGA Slaves Region

The lightweight FPGA slaves provide access to slaves in the FPGA fabric through the lightweight HPS-to-FPGA bridge.

Peripherals Region

The peripherals region is near the top of the address space. The peripherals region includes slaves connected to the L3 interconnect and L4 buses.

On-Chip RAM Region

The on-chip RAM is always mapped near the top of the address space, independent of the boot region contents.

Boot ROM Region

The boot ROM is always mapped near the top of the address space, independent of the boot region contents.

SCU and L2 Registers Region

The SCU and L2 registers region provides access to internally-decoded MPU registers (SCU and L2).

SDRAM Address Space

The SDRAM address space is up to 4 GB. The entire address space can be accessed through the FPGA‑to‑SDRAM interface from the FPGA fabric. The total amount of SDRAM addressable from the other address spaces varies.

There are cacheable and non-cacheable views into the SDRAM space. When a master of the system interconnect performs a cacheable access to the SDRAM, the transaction is performed through the ACP port of the MPU subsystem. When a master of the system interconnect performs a non-cacheable access to the SDRAM, the transaction is performed through the 32-bit main L3 interconnect master of the SDRAM L3 interconnect.

Address Remapping

The system interconnect supports address remapping through the noc_addr_remap_value, noc_addr_remap_set, and noc_addr_remap_clear registers in the system manager. Remapping allows software to control which memory device (on-chip RAM or boot ROM) is accessible at address 0x0. The remap registers can be modified by the MPU and the FPGA-to-HPS bridge.

The following master interfaces are affected by the remap bits:

MPU master interface
- L2 cache master 0 interface
Non-MPU master interfaces
- DMA master interface
- Master peripheral interfaces
- Debug Access Port (DAP) master interface
- FPGA-to-HPS bridge master interface

Memory Map Remap Bits

Table 47. Remap Bit Usage
`remap0`	`remap1`	0x00000000 (MPU Master Interface)	0x00000000 (Non-MPU Master Interfaces)	0xFFE00000 (All Maps)	0xFFFC0000 (MPU & DAP)
0	0	Boot ROM	SDRAM	On-Chip RAM	Boot ROM
0	1	Invalid setting
1	0	SDRAM	SDRAM	On-Chip RAM	Boot ROM
1	1	On-Chip RAM	On-Chip RAM	On-Chip RAM	Boot ROM

L2 filter registers in the MPU subsystem, not the interconnect, allow the SDRAM to be remapped to address 0x0 for the MPU.

Secure Transaction Protection

The system interconnect provides two levels of secure transaction protection:

Security firewalls—Enforce secure read and write transactions.
Privilege filter—Leverages the firewall mechanism and provides additional security by filtering the privilege level of L4 slave transactions. The privilege filter applies to writes only.

All slaves on the SoC are placed behind a security firewall. A subset of slaves are also placed behind a privilege filter. Transactions to these slaves must pass both a security firewall and the privilege filter to reach the slave.

System Interconnect Master Properties

The system interconnect connects to slave interfaces through the main L3 interconnect and SDRAM L3 interconnect.

Table 48. System Interconnect Master Interfaces
Master	Interface Width	Clock	Security	SCR¹¹ Access	Privilege	Issuance (Read/Write/Total)	Type
MPU Subsystem L2 cache M0/1	64	mpu_l2ram_clk	Per Transaction	Yes	Transaction based	7/12/23	AXI*
FPGA-to-HPS Bridge ¹²	32/64/128	fpga2hps_clk	Per Transaction	Yes	Transaction based	8/8/8	AXI*
FPGA-to-SDRAM Ports ¹²	32/64/128	f2h_sdram_clk[2:0]	Per Transaction	No	Transaction based	8/8/8	AXI*
DMA	64	l4_main_clk	Per Transaction	No	User mode	8/8/8	AXI*
EMAC 0/1/2	32	l4_mp_clk	Secure/Non-Secure	No	User mode	16/16/32	AXI*
USB OTG 0/1	32	l4_mp_clk	Nonsecure	No	User mode	2/2/4	AHB*
NAND	32	l4_mp_clk	Nonsecure	No	User mode	1/1/2	AXI*
SD/MMC	32	l4_mp_clk	Nonsecure	No	User mode	2/2/4	AHB*
ETR	32	cs_at_clk	Per Transaction	No	Transaction based	32/1/32	AXI*
AHB* -AP	32	l4_mp_clk	Per Transaction	Yes	Transaction based	1/1/1	AHB*

Master Caching and Buffering Overrides

Some of the peripheral masters connected to the system interconnect do not have the ability to drive the caching and buffering signals of their interfaces. The system manager provides registers so that you can enable cacheable and bufferable transactions for these masters. The system manager drives the caching and buffering signals of the following masters:

Master Peripheral	System Manager Registers
EMAC0, EMAC1, and EMAC2	`emac0`, `emac1`, and `emac2`
USB OTG 0 and USB OTG 1	`usb0_l3master` and `usb1_l3master`
NAND flash	`nand_l3master`
SD/MMC	`sdmmc_l3master`

At reset time, the system manager drives the cache and buffering signals for these masters low. In other words, the masters listed do not support cacheable or bufferable accesses until you enable them after reset. There is no synchronization between the system manager and the system interconnect, so avoid changing these settings when any of the masters are active.

System Interconnect Slave Properties

The system interconnect connects to various slave interfaces through the main L3 interconnect, the SDRAM L3 interconnect, and the L4 peripheral buses. After reset, all slave interfaces are set to the secure state.

Table 49. System Interconnect Slave Interfaces
Slave	Interface Width	Clock	Acceptance (Read/Write/Total)¹³	Security	Privilege	Interface Type
SP Timer 0/1/2/3	32	l4_sp_clk	1/1/1	Boot Secure ¹⁴	User Mode	APB* ¹⁵
I2C 0/1/2/3/4	32	l4_sp_clk	1/1/1	Boot Secure	User Mode	APB* ¹⁵
UART 0/1	32	l4_sp_clk	1/1/1	Boot Secure	User Mode	APB* ¹⁵
GPIO 0/1/2	32	l4_sp_clk	1/1/1	Boot Secure	User Mode	APB* ¹⁵
SD/MMC CSR	32	l4_mp_clk	1/1/1	Boot Secure	User Mode	APB* ¹⁵
EMAC 0/1/2	32	l4_mp_clk	1/1/1	Boot Secure	User Mode	APB* ¹⁵
OSC Timer 0/1/2/3	32	l4_sys_free_clk	1/1/1	Secure ¹⁶	Privileged	OCP
Watchdog 0/1/2/3	32	l4_sys_free_clk	1/1/1	Secure	Privileged	APB* ¹⁵
Clock Manager	32	l4_sys_free_clk	1/1/1	Secure	Privileged	OCP
Reset Manager	32	l4_sys_free_clk	1/1/1	Secure	Privileged	OCP
System Manager	32	l4_sys_free_clk	1/1/1	Secure	Privileged	OCP
FPGA Manager Data	32	l4_sys_free_clk	1/1/1	Secure	Privileged	OCP
FPGA Manager CSR	32	l4_sys_free_clk	1/1/1	Secure	Privileged	OCP
DAP	32	l4_sys_free_clk	1/1/1	Secure	Privileged	APB* ¹⁵
DMA Secure CSR	32	l4_main_clk	1/1/1	Boot Secure	User Mode	APB* ¹⁵
DMA Non-Secure CSR	32	l4_main_clk	1/1/1	Boot Secure	User Mode	APB* ¹⁵
SPI Slave 0/1	32	l4_main_clk	1/1/1	Boot Secure	User Mode	APB* ¹⁵
SPI Master 0/1	32	l4_main_clk	1/1/1	Boot Secure	User Mode	APB* ¹⁵
USB OTG CSR 0/1	32	l4_mp_clk	1/1/1	Boot Secure	User Mode	AHB*
NAND CSR	32	l4_mp_clk	1/1/1	Boot Secure	User Mode	AHB*
NAND Command and Data	32	l4_mp_clk	1/1/1	Boot Secure	User Mode	AHB*
SD/MMC ECC	32	l4_mp_clk	1/1/2	Secure	Privileged	OCP
On Chip RAM ECC	32	l4_mp_clk	1/1/2	Secure	Privileged	OCP
DMA ECC	32	l4_mp_clk	1/1/2	Secure	Privileged	OCP
NAND ECC	32	l4_mp_clk	1/1/2	Secure	Privileged	OCP
USB OTG ECC	32	l4_mp_clk	1/1/2	Secure	Privileged	OCP
EMACS ECC	32	l4_mp_clk	1/1/2	Secure	Privileged	OCP
ACP	64	mpu_l2ram_clk	13/5/18	Secure	User Mode	AXI*
APB* -DP	32	cs_pl4	1/1/1	Secure	Privileged	APB* ¹⁵
DDR	256	f2h_sdram_clk[2:0]	16/16/16	Secure/Non-Secure	User Mode	Avalon
Lightweight HPS-to-FPGA Bridge	32	lwh2fpga_clk	8/8/8	Boot Secure	User Mode	AXI*
HPS-to-FPGA Bridge	128/64/32	hps2fpga_clk	8/8/8	Boot Secure	User Mode	AXI*
On-Chip RAM	64	l3_main_free_clk	2/2/2	Secure/Non-Secure Per 4KB region	User Mode	AXI*
STM	32	cs_at_clk	1/2/2	Secure/Non-Secure	User Mode	AXI*

Note: APB* -DP has no direct connection to the system interconnect.

System Interconnect Clocks

The system interconnect clocks are driven by the clock manager. The system interconnect's clocks are part of the NoC clock group, which is hardware-sequenced.

List of Main L3 Interconnect Clocks

Table 50. Main L3 Interconnect Clocks
Clock Name	Description	Synchronous to l3_main_free_clk
`l3_main_free_clk`	Clocks the main L3 interconnect	—
`l4_sys_free_clk`	Clocks the following interconnect components: The L4 system bus The interface to the DAP	Y
`l4_main_clk`	Clocks fast L4 peripherals on the L4 main bus Refer to "System Interconnect Master Properties" and "System Interconnect Slave Properties" for detailed peripheral-to-clock mappings.	Y
`l4_mp_clk`	Clocks the following interconnect components: Mid-speed L4 peripherals on the L4 MP bus The L4 AHB* bus The L4 ECC bus Refer to "System Interconnect Master Properties" and "System Interconnect Slave Properties" for detailed peripheral-to-clock mappings.	Y
`l4_sp_clk`	Clocks slow L4 peripherals on the L4 SP bus. Refer to "System Interconnect Master Properties" and "System Interconnect Slave Properties" for detailed peripheral-to-clock mappings.	Y
`cs_at_clk`	CoreSight* trace clock. Clocks the CoreSight* Embedded Trace Router (ETR) master interface.	Y
`mpu_l2ram_clk`	Clocks the MPU subsystem master interfaces.	N
`fpga2hps_clk`	Clocks the FPGA-to-HPS bridge.	N
`hps2fpga_clk`	Clocks the HPS-to-FPGA bridge.	N
`lwh2fpga_clk`	Clocks the lightweight HPS-to-FPGA bridge.	N

List of SDRAM L3 Interconnect Clocks

Table 51. SDRAM L3 Interconnect Clocks
Clock Name	Description	Synchronous to l3_main_free_clk
`hmc_free_clk`	Clock from the hard memory controller. Clocks the SDRAM L3 interconnect. The `hmc_free_clk` frequency is ½ the interface clock frequency.	N
`f2s_sdram_clk [2:0]`	Clocks the FPGA-to-SDRAM interfaces.	N

System Interconnect Resets

The system interconnect has one reset signal, l3_rst_n. The reset manager drives this signal to the system interconnect on a cold or warm reset.

Functional Description of the Rate Adapters

Rate adapters are used at points in the system interconnect where there are bandwidth discontinuities, to ensure efficient use of interconnect data pathways. They are placed where a low-bandwidth source feeds a high-bandwidth destination. For this reason, they are sometimes positioned at the response side (where the Master is faster) and sometimes at the request side (where the Slave is faster).

The following table shows the rate adapters.

Table 52. Rate Adapter Modules
Rate Adapter Module Name	Data Path
noc_mpu_m0_MPU_M1toDDRResp_main_RateAdapter	Data packets from the SDRAM L3 interconnect in response to the MPU
noc_mpu_m0_MPU_M0_rate_adResp_main_RateAdapter	Data packets from the L3 interconnect in response to the MPU
noc_mpu_m0_L4_MP_rate_ad_main_RateAdapter	Data packets carrying requests from L4 master peripherals to the L3 interconnect
noc_mpu_m0_fpga2soc_rate_ad_main_RateAdapter	Data packets carrying requests from the FPGA-to-HPS bridge master to the L3 interconnect
noc_mpu_m0_L3Tosoc2fpgaResp_main_RateAdapter	Data packets from the HPS-to-FPGA and lightweight HPS-to-FPGA bridges in response to the L3 interconnect
noc_mpu_m0_acp_rate_ad_main_RateAdapter	Data packets carrying requests from the L3 interconnect to the ACP

Functional Description of the Firewalls

Security

Slave Security

The system interconnect enforces security through the slave settings. The slave settings are controlled by the NoC Security Control Register (SCR) in the service network. Each L3 and L4 slave has its own security check and programmable security settings. After reset, every slave of the system interconnect is set to a secure state (referred to as boot secure). Only secure masters are allowed to access secure slaves.

The NoC implements five firewalls to check the security state of each slave, as listed in the following table. At reset time, all firewalls default to the secure state.

Table 53. NoC Firewalls
Name	Function
On-Chip RAM Firewall	Filter access to on-chip RAM
Peripherals Firewall	Filter access to slave peripherals (SPs) in the following buses: L4 main bus L4 master peripherals bus L4 AHB* bus L4 slave peripherals bus
System Firewall	Filter access to system peripherals in the following components: L4 system bus L4 ECC bus DAP
HPS-to-FPGA Firewall	Filter access to FPGA through the following bridges: HPS-to-FPGA bridge Lightweight HPS-to-FPGA bridge
DDR and DDR L3 Firewalls	Filter access to DDR SDRAM

At reset, the privilege filters are configured to allow certain L4 slaves to receive only secure transactions. Software must either configure bridges secure at startup, or reconfigure the privilege filters to accept nonsecure transactions. You can reconfigure the privilege filters through the l4_priv register in the noc_l4_priv_l4_priv_filter module.

To change the security state, you must perform a secure write to the appropriate SCR register of a secure slave. A nonsecure access to the SCR register of a secure slave triggers a response with random data.

Note: Future devices might not support the return of random data and might only support an error response for blocked firewall transactions. For designs that may be ported to future devices, Intel recommends you to set the error_response bit in the global register of the noc_fw_ddr_l3_ddr_scr module.

Master Security

Masters of the system interconnect are either secure, nonsecure, or the security is set on a per transaction basis. L2 cache masters 0 and 1, the FPGA-to-HPS bridge, DMA, the Ethernet MACs, and the DAP perform secure and nonsecure accesses on a per-transaction basis. All other system interconnect masters perform nonsecure accesses.

Accesses to secure slaves by unsecure masters result in a response with random data.

Functional Description of the SDRAM L3 Interconnect

The SDRAM L3 interconnect consists of two main blocks, serving the following two main functions:

The SDRAM scheduler provides multi-port scheduling between the SDRAM L3 interconnect masters and the hard memory controller in the FPGA portion of the SoC. The SDRAM L3 interconnect is mastered by the MPU, the main L3 interconnect, and FPGA-to-SDRAM ports.
The SDRAM adapter provides connectivity between the SDRAM L3 interconnect masters and the hard memory controller.

The SDRAM L3 interconnect also includes firewalls that can protect regions of SDRAM from unauthorized access.

The hard memory controller is physically located in the FPGA portion of the device, and therefore it is in a separate power domain from the HPS. The HPS cannot use the SDRAM L3 interconnect until the FPGA portion is powered up.

Functional Description of the SDRAM Scheduler

The SDRAM scheduler functions as a multi-port front end (MPFE), scheduling transactions from multiple masters to the SDRAM.

The SDRAM scheduler manages transactions to the memory access regions in the SDRAM. These memory regions are defined by the SDRAM L3 firewalls. The second-stage bootloader is expected to program the scheduler with the correct timings to implement optimal access patterns to the hard memory controller.

The SDRAM scheduler has the following features:

Five input connections
- One 64-bit connection from the MPU
- One 64-bit connection from the main L3 interconnect
- Two 128/64/32-bit connections from the FPGA
- One 64/32-bit connection from the FPGA
Single 256-bit connection to the SDRAM L3 adapter
- Capable of issuing transactions at the memory device line rate
- Traffic is comprised of aggregate inputs

Monitors for Mutual Exclusion

The SDRAM scheduler implements support for mutually-exclusive (mutex) accesses on all ports to the SDRAM L3 interconnect.

The process for a mutually-exclusive access is as follows:

A master attempts to lock a memory location by performing an exclusive read from that address.
The master attempts to complete the exclusive operation by performing an exclusive write to the same address location.
The exclusive write access is signaled as:
- Failed if another master has written to that location between the read and write accesses. In this case the address location is not updated.
- Successful otherwise.

To support mutually-exclusive accesses from the MPU, the memory must be configured in the MMU page tables as normal memory, shareable, or non-cacheable.

Exclusive Access Support

To ensure mutually exclusive access to shared data, use the exclusive access support built into the SDRAM scheduler. The AXI buses that interface to the scheduler provide ARLOCK[0] and AWLOCK[0], signals which are used by the scheduler to arbitrate for exclusive access to a memory location. The SDRAM scheduler contains six monitors. Each monitor can be used by any of the following exclusive-capable masters:

CPU 0
CPU 1
FPGA-to-HPS bridge
FPGA-to-SDRAM0 port
FPGA-to-SDRAM1 port
FPGA-to-SDRAM2 port

Each master can lock only one memory location at a time.

Arbitration and Quality of Service in the SDRAM Scheduler

The SDRAM scheduler controls arbitration priorities internally for optimal end-to-end quality of service. You can apply QoS settings to each port of the SDRAM scheduler to control the bandwidth available to each master.

Each master on the SDRAM scheduler has software-programmable QoS signals. These signals are propagated to the scheduler and used as arbitration criteria for access to SDRAM.

For information about programming quality of service for the FPGA-to-SDRAM masters, refer to "Functional Description of the QoS Generators" and "Configuring the Quality of Service Logic".

Functional Description of the SDRAM Adapter

The SDRAM adapter connects the SDRAM scheduler with the Hard Memory Controller.

The SDRAM adapter provides the following functionality:

ECC generation, detection, and correction
Operates at memory half rate
- Matches interface frequency of the single port memory controller in the FPGA
- Connectivity to the MPU, main L3 interconnect, and FPGA undergo clock crossing

ECC

The SDRAM Adapter ECC can detect and correct single-bit errors and detect double-bit errors.

The addresses are merged with data and are used for checking. This configuration detects if correct write data is written to an incorrect location in memory; and if correct data is read from an incorrect location in memory.

Note: Data and address errors are detected independently.

ECC Write Behavior

When data is written to SDRAM, the SDRAM controller generates an ECC based on the write data and the write address.

If the write transaction is a partial write (less than 64 bits wide), the SDRAM adapter implements it as a read-modify-write (RMW) sequence, as follows:

Reads existing data from the specified address
Combine the write data with the existing data
Generates the ECC based on the combined data and the write address
Writes the combined data back to the write address

ECC Read Behavior

When data is read from SDRAM, the SDRAM controller checks the ECC to determine if the data or address is incorrect. It handles the following cases:

If the SDRAM controller finds a single-bit data error, it corrects in the data returned to the master. You can also enable the SDRAM adapter to write the corrected data back to memory.
If the SDRAM controller finds a double-bit data error, the SDRAM L3 interconnect issues a bus error. The SDRAM L3 interconnect can also issue an interrupt, if enabled. Double-bit errors cannot be corrected.
If the SDRAM controller finds an error in the address, indicating an address bit upset between the adapter and the HMC, the SDRAM L3 interconnect hardware issues a bus error.

SDRAM Adapter Interrupt Support

The SDRAM adapter supports the following three interrupts:

The status interrupt occurs when:
- Calibration is complete.
- The ECC is unable to schedule an auto-correction write-back to memory. This occurs only when the auto-write-back FIFO is full.
The ECC read-back interrupt occurs when an ECC single-bit error is detected in the read data. When this happens, the return data is corrected and returned to the NoC.
The double-bit or fatal error interrupt occurs when any of the following three errors happens:
- A double-bit error in the read data has been detected, which cannot be corrected.
- A single-bit error has been detected in the address field. This means that the data that the adapter is returning has no bit errors, but is not the requested data. When this happens, the adapter returns a data error along with the data.
- Any of the DDR4 devices have triggered their ALERT pins.
  - Address or command parity check has failed
  - Write data CRC check has failed
  - Cannot gracefully recover because SDRAMs are not providing feedback on failure case

SDRAM Adapter Clocks

All the logic in the SDRAM adapter is synchronous and effectively running off a single clock, hmc_free_clk, which is provided by the hard memory controller.

SDRAM L3 Firewalls

All data that is routed to the SDRAM scheduler must pass through the firewalls.

The SDRAM L3 firewalls define memory access regions in the SDRAM. Each SDRAM L3 interconnect master has its own memory access regions, independent of the other masters. The firewalls define whether each memory access region is protected or unprotected relative to its master. The number of available memory access regions for each master is shown in the following table.

Table 54. Memory Access Regions for SDRAM Masters
SDRAM L3 Interconnect Master	Number of Memory Access Regions
MPU	4
Main L3 interconnect (including the FPGA-to-HPS bridge)	8
FPGA-to-SDRAM port 0	4
FPGA-to-SDRAM port 1	4
FPGA-to-SDRAM port 2	4

The SDRAM L3 interconnect regulates access to the hard memory controller with the firewalls, which support secure regions in the SDRAM address space. Accesses to the SDRAM pass through the firewalls and then through the scheduler.

SDRAM L3 Interconnect Resets

The reset signal l3_rst_n resets the system interconnect and the SDRAM L3 interconnect, but not the hard memory controller.

The reset signal in the hard memory controller is automatically connected to the SDRAM L3 interconnect when you instantiate the HPS component. For information about resetting the hard memory controller, refer to the External Memory Interfaces in Arria 10 Devices chapter of the Arria 10 Core Fabric and General Purpose I/O Handbook.

Soft logic in the FPGA must support the global_reset_n signal correctly. Refer to the Instantiating the HPS Component chapter for information about global_reset_n.

To optionally preserve the contents of the SDRAM on reset, refer to "Reset Handshaking" in the Reset Manager chapter.

Figure 37. Recommended SDRAM Reset Connections

Note: It is important to connect the reset user logic directly to both the HPS and the hard memory controller. If the hard memory controller is reset while the HPS is still running, the HPS is unable to access any external SDRAM memory.

Functional Description of the Arbitration Logic

The interconnect arbitration logic handles situations in which multiple simultaneous packets need the same interconnect resource.

The system interconnect contains arbitration nodes at each point where multiple packets might demand the same resource. Each arriving packet has a priority. When multiple packets of different priorities arrive simultaneously at a node, the arbitration logic grants the resource to the highest-priority packet.

If there are simultaneous packets with the same priority, the system interconnect uses a simple round-robin (least recently used) algorithm to select the packet.

Each packet's priority is determined by urgency, pressure, and hurry, as described in "QoS Mechanisms".

Functional Description of the QoS Generators

Quality of service is managed by information generated by a QoS generator at each initiator interface (master). The QoS generators work with the arbitration logic so you can tune the interconnect's performance to your system needs.

At the entry points to the system interconnect, the QoS generators create the following information to control how the arbitration logic allocates interconnect bandwidth among all transactions:

Urgency—Each master has an urgency value. When it initiates a packet, it assigns a priority equal to its urgency.
Pressure—Each data path has a pressure value. If the pressure is raised, packets on that path are treated as if their priority was also raised.
Hurry—Each master has a hurry value. If the hurry is raised, all packets from that master are treated as if their priority was also raised.

The QoS information is used at each interconnect arbitration node to assign a packet priority. When there is contention for a resource, the arbitration logic generally gives the resource to the packets with higher priority.

For detailed descriptions of urgency, pressure, and hurry, refer to "QoS Mechanisms".

The values of urgency, pressure, and hurry are determined by each QoS generator's operating mode. QoS generators support the following modes:

Limiter
Regulator
Fixed

For detailed descriptions of the QoS generator modes, refer to "QoS Generator Modes".

QoS Generators in the Interconnect Architecture

The principal masters into the system interconnect are equipped with QoS generators.

The following table shows the interconnect initiators (masters) that support QoS generation.

Table 55. QoS Generators on Interconnect Masters
Master Name(s)	Default QoS Generator Mode	Clock
MPU Subsystem L2 cache M0/1	Limiter	`mpu_l2ram_clk`
FPGA-to-HPS Bridge	Limiter	`fpga2hps_clk`
FPGA-to-SDRAM Ports	Regulator	`f2h_sdram_clk`
DMA	Limiter	`l4_main_clk`
EMAC 0/1/2	Regulator	`l4_mp_clk`
USB OTG 0/1	Regulator	`l4_mp_clk`
NAND	Regulator	`l4_mp_clk`
SD/MMC	Regulator	`l4_mp_clk`

QoS Mechanisms

The QoS generators influence arbitration priorities by manipulating urgency, pressure, and hurry.

When two or more packets compete for an interconnect resource at an arbitration node, the arbitration logic selects the packet with the highest priority. QoS generators can control the behavior of the arbitration logic by modifying the priority of packets through urgency, pressure, and hurry.

The QoS controls for each master are separated into read and write QoS controls.

Packet Urgency

An urgency value is attached to each packet as long as it is moving through the interconnect. Urgency is the primary QoS mechanism.

A QoS generator assigns an urgency level to each packet when the packet data arrives at an interconnect master port.

Pressure Propagation

Each active datapath in the interconnect has a pressure level, which is the highest urgency of all packets currently in the datapath. Pressure can change depending on the traffic through the datapath.

The pressure is based on the urgency of packets waiting for that datapath. If a higher-priority packet requests a datapath that is already active, the datapath’s urgency increases.

When a packet arrives at an arbitration node, normally its priority is the urgency assigned by its master. However, if the datapath's pressure is greater than the urgency, that pressure tells the node that there is a higher-urgency packet somewhere behind the current one. This prompts the node to expedite the lower-urgency packet to free up the datapath.

For example, consider three active packets in the interconnect. Packets A and B are moving through the same datapath, with A ahead of B. Packet C is on a different datapath.

Packet A's urgency value is 1, and packet B's urgency is 3. Packet A arrives at a particular node simultaneously with Packet C—and Packet C's urgency is 2. Without the pressure mechanism, Packet C would pass through the node first, ahead of Packet B, even though Packet B has the highest urgency of the three packets.

However, because Packet B's urgency is 3, its datapath has a pressure of 3. That pressure value tells the node to accept the packet on that datapath—Packet A—even though it has a lower urgency. After Packet A is out of the way, the node can accept Packet B ahead of Packet C.

Hurry

Each QoS generator has a hurry level, which can change depending on the bandwidth usage and the QoS generator mode.

Hurry is a mechanism to propagate the master's urgency level to all nodes that receive packets from that master. Hurry allows the master to influence the priority not only of the current transaction, but also of all its transactions that are still active.

When the hurry level is increased at a master, it has the effect of setting the urgency of that master’s transactions to at least the same level as the hurry. As a result, the targets respond to the transactions more quickly.

For example, consider a master whose QoS generator is configured in regulator mode. This QoS generator dynamically raises and lowers its urgency value depending on the traffic it experiences. Packets sent from the master use the QoS generator's current urgency value. If the QoS generator raises its urgency value, it uses the Hurry mechanism to increase the pressure value at every datapath where it has a pending transaction.

The QoS generator implements hurry by broadcasting an urgency packet on every datapath where it has pending transactions. The urgency packet contains the current urgency value, which raises pressure on each datapath.

QoS Generator Modes

Each QoS generator can operate in any of the following modes:

Limiter—Each initiator has a maximum bandwidth cap to prevent it from sending too many requests. When the cap is reached, the initiator stops accepting new requests.
Regulator—Each initiator has an ideal minimum bandwidth threshold. When the initiator's bandwidth usage drops below threshold, it raises the urgency for new transactions. Any pending transactions from that initiator are also expedited by the hurry mechanism.
Fixed—Each initiator assigns static urgency values to packets. The urgency value for read packets can be different from the value for write packets.

QoS generators can also be set to bypass mode. In this case, QoS signals are provided by the component connected to the master interface.

For the default QoS generator mode for each initiator, refer to "QoS Generators in the Interconnect Architecture".

Limiter Mode

In limiter mode, transactions from the initiator are limited to a specific bandwidth.

The QoS generator controls the initiator's access to the interconnect, by monitoring the amount of data transmitted and received in a sliding time window.

The purpose of limiter mode is to place a hard bound on the initiator's throughput. This avoids flooding the interconnect with transactions from that initiator. When a programmable bandwidth threshold is exceeded, the initiator is stalled, and stops accepting new requests. As the time window moves forward, the total bandwidth usage in the window falls, and the initiator resumes accepting requests.

In limiter mode, the QoS generator combines read and write requests to determine the bandwidth usage.

Regulator Mode

In regulator mode, the QoS generator maintains bandwidth usage to transaction targets at a programmed level, by adjusting urgency, pressure, and hurry.

Use this mode to guarantee a specific bandwidth is available to the master, while preventing it from claiming more bandwidth than you requested.

When the initiator's bandwidth usage exceeds the specified threshold, the QoS downgrades the urgency, pressure, and hurry levels. When bandwidth usage is too low, the QoS increases urgency, pressure and hurry. Software can program the desired bandwidth threshold, the saturation, and maximum and minimum levels for urgency, pressure, and hurry.

Regulator mode controls bandwidth usage more smoothly than limiter mode. This mode is useful in achieving real-time performance.

Fixed Mode

In fixed mode, the QoS generator assigns a fixed urgency to each packet. Software can configure separate urgencies for read and write transactions.

This mode is useful for coarse-grain control in a system where one master should have the first access to interconnect resources. However, this mode can have significant system level performance degradation due to other masters being starved.

Functional Description of the Observation Network

Probes

The system interconnect includes the probes shown in the following table.

Table 56. Observation Probes
Probe Name	Probe ID	Probe Type	Probe Target(s)	Packet Tracing and Statistics	Transaction Profiling	Error Logging
MPU	6	Packet	MPU master 0	•
SOC2FPGA	5	Packet	HPS-to-FPGA bridge Lightweight HPS-to-FPGA bridge	•
DDR	10	Packet	SDRAM adapter	•
EMAC	4	Packet	EMAC0		•
EMAC	4	Packet	EMAC1 EMAC2	•
ERROR	n/a	Error	HPS-to-FPGA bridge Lightweight HPS-to-FPGA bridge`			•

The probe ID is available on the ATB trace bus.

Packet Tracing, Profiling, and Statistics

Packet probes perform packet tracing, profiling and statistics. The following table shows how each packet probe is configured. All statistics counters are 16 bits wide, and all probes can generate level alarms from the statistics counter. All probes use the CoreSight cross-trigger protocol.

Table 57. Packet Probe Characteristics
Probe Name	Purpose	Number of Filters	Statistics Counters Can Count Enabled Bytes	Packet Payloads Included in Trace Data	Number of Statistics Counters
MPU	MPU	1	No	No	4
SOC2FPGA	HPS-to-FPGA bridge	2	No	No	4
EMAC	Ethernet MAC	2	No	No	4
DDR	SDRAM	4	Yes	Yes	2

Packet Filtering

You can set up filters to control how the observation network handles traced packets.

Filters can perform the following tasks:

Select which packets the observation network routes to CoreSight*
Trigger a trace alarm when a packet meets specified criteria

Statistics Collection

To collect packet statistics, you specify packet criteria and set up counters for packets that meet those criteria. You can set up the observation network to trigger an alarm when a counter reaches a specified level.

Transaction Profiling

A transaction probe is available on Ethernet MAC 0. You can use the transaction probe to measure either the transaction latency or the number of pending packets on EMAC0. Data are collected as a histogram.

The EMAC0 transaction probe is configured as shown in the following table.

Table 58. EMAC0 Transaction Probe Configuration
Number of simultaneously tracked transactions	4
Width of counters	10 bits
Available delay thresholds	64, 128, 256, 512
Available pending transaction count thresholds	2, 4, 8
Number of comparators	3
Number of histogram bins	4

Profiling Transaction Latency

In latency mode (also called delay mode), one of the four delay threshold values can be chosen for each comparator. The threshold values represent the number of clock cycles that a transaction takes from the time the request is issued to the time the response is returned.

Profiling Pending Transactions

In pending transaction mode, one of the three available transaction count threshold values can be chosen for each comparator. The threshold values represent the number of requests pending on EMAC0.

Packet Alarms

Packet alarms can be used to trigger software interrupts.

The following types of alarms are available:

Trace alarms—You can examine trace alarms through the system interconnect registers.
Statistics alarms

Error Logging

The error probe logs errors on initiators, targets, and firewalls.

Configuring the System Interconnect

Configuring the Rate Adapters

The default settings for the rate adapters are as shown in the following table.

Table 59. Rate Register Default Settings
Rate Register Name	Default
`MPU_M1toDDRResp_main_RateAdapter_Rate`	0x0
`MPU_M0_rate_adResp_main_RateAdapter_Rate`	0x0
`L4_MP_rate_ad_main_RateAdapter_Rate`	0x100
`fpga2soc_rate_ad_main_RateAdapter_Rate`	0x0
`L3Tosoc2fpgaResp_main_RateAdapter_Rate`	0x0
`acp_rate_ad_main_RateAdapter_Rate`	0x0

Configuring the SDRAM Scheduler

FPGA Port Configuration

The FPGA has three outputs that pass through the firewall before connecting to the SDRAM scheduler.

You can configure the FPGA-to-SDRAM (F2SDRAM) ports to data widths of 32, 64, or 128 bits:

F2SDRAM 0 - 32-, 64-, or 128-bit data widths
F2SDRAM 1 - 32- or 64-bit data widths
F2SDRAM 2 - 32-, 64-, or 128-bit data widths

There are four port configurations that are a combination of the SDRAM ports 0 - 2 that you are able to select. Once a port configuration is selected, you can choose to disable ports that you do not need.

Note: The total data width of all interfaces is limited to a maximum of 256 bits in the read direction and 256 bits in the write direction.

Port Configuration	F2SDRAM 0	F2SDRAM 1	F2SDRAM 2
1	32-bit	32-bit	32-bit
2	64-bit	64-bit	64-bit
3	128-bit	unused	128-bit
4	128-bit	32-bit	64-bit

Memory Timing Configuration

The SDRAM L3 interconnect provides fully-programmable timing parameter support for all JEDEC-specified timing parameters.

The following lists the handoff information used to control the SDRAM scheduler:

The scheduler is aware of the SDRAM timing so that it can guide traffic into the hard memory controller.
This information is not used to control the subsystem that sets up memory timings in the hard memory controller hardware.

Configuring SDRAM Burst Sizes

You can optimize memory throughput by adjusting the SDRAM burst lengths for read and write transactions. The best burst length values are application-dependent. Intel recommends that you follow guidelines in Arria 10 SoC Device Design Guidelines to determine the optimal burst lengths.

Configuring the Hard Memory Controller

SDRAM Adapter Memory Mapped Registers

The SDRAM adapter memory mapped registers (MMRs) are used for configuring and reading ECC status; and configuring data width adaptation for 16-, 32-, and 64-bit data widths.

Sharing I/Os between the EMIF and the FPGA

Arria 10 SoC devices have three modular I/O banks to connect the HPS to an SDRAM (2K, 2J, and 2I) through a dedicated HPS EMIF.

Each bank has four I/O lanes that correspond to:

Lane 3: IO[47:36]
Lane 2: IO[35:24]
Lane 1: IO[23:12]
Lane 0: IO[11:0]

Figure 38. HPS Hard Memory Controller

To use SDRAM, you instantiate the HPS EMIF in Platform Designer, by selecting the Arria 10 External Memory Interfaces for HPS component. Quartus Prime software assigns the correct banks and lanes for the SDRAM I/O.

I/Os in these three banks can be shared between the HPS EMIF and FPGA. Quartus Prime software enforces the following guidelines on shared I/Os:

Modular I/O banks 2K, 2J and 2I can be used entirely as FPGA GPIO when there is no HPS EMIF in the system.
Bank 2K:
- If there is an HPS EMIF in the system, lane 3 is used for ECC for the SDRAM. Unused pins in lane 3 may be used as FPGA inputs only.
- Lanes 2, 1, and 0 are used for address and command for the SDRAM. Unused pins in these lanes may be used as FPGA inputs or outputs.
Bank 2J:
- If there is an HPS EMIF in the system, bank 2J is used for data bits [31:0].
  - With 16-bit data widths, unused pins in the two lanes of bank 2J used for data can be used as inputs only. The pins in the remaining two lanes can be used as FPGA inputs or outputs.
  - With 32-bit data widths, unused pins bank 2J can be used as FPGA inputs only.
Bank 2I:
- If there is an HPS EMIF in the system, bank 2I is used for data bits [63:32]
  - With 64-bit data widths, unused pins in bank 2I can be used as FPGA inputs only.
  - With 32- or 16-bit data widths, bank 2I can be used as FPGA inputs or outputs.

Configuring the Quality of Service Logic

You can programmatically configure the QoS generator for each master through the QoS registers.

Note: Before accessing the registers for any QoS generator, you must ensure that the corresponding peripheral is not in reset. Otherwise, the register access results in a bus error, causing a CPU fault.

Programming QoS Limiter Mode

QoS bandwidth limiter mode is mode 1. To put the QoS generator in limiter mode, set the registers as shown in the following table.

Table 60. QoS Generator Register Values for Limiter Mode
Register	Field	Value
`I_main_QosGenerator_Mode`	`MODE`	1
`I_main_QosGenerator_Priority`	`P0`	Urgency for write transactions
`I_main_QosGenerator_Priority`	`P1`	Urgency for read transactions
`I_main_QosGenerator_Bandwidth`	`BANDWIDTH`	Maximum bandwidth, in units of (bytes/cycle)/256
`I_main_QosGenerator_Saturation`	`SATURATION`	Measurement window for bandwidth, in units of bytes/16. This register specifies the maximum number of bytes that the initiator can transmit or receive at full speed before the limiter triggers.

Higher priority (urgency) values mean that a packet receives preferential treatment at each arbitration node.

For detailed information about setting bandwidth and saturation, refer to "Bandwidth and Saturation".

When you switch QoS modes, the bandwidth counter is reset.

Programming QoS Regulator Mode

QoS bandwidth regulator mode is mode 3. To put the QoS generator in regulator mode, set the registers as shown in the following table.

Table 61. QoS Generator Register Values for Regulator Mode
Register	Field	Value
`I_main_QosGenerator_Mode`	`MODE`	3
`I_main_QosGenerator_Priority`	`P0`	Packet urgency when the actual throughput exceeds the threshold set in `I_main_QosGenerator_Bandwidth`.
`I_main_QosGenerator_Priority`	`P1`	Packet urgency when the actual throughput is less than the threshold set in `I_main_QosGenerator_Bandwidth`. `P0` must be less than or equal to `P1`.
`I_main_QosGenerator_Bandwidth`	`BANDWIDTH`	Desired throughput, in units of (bytes/cycle)/256
`I_main_QosGenerator_Saturation`	`SATURATION`	Measurement window for bandwidth, in units of bytes/16.

Higher priority (urgency) values mean that a packet receives preferential treatment at each arbitration node.

For detailed information about setting bandwidth and saturation, refer to "Bandwidth and Saturation".

When you switch QoS modes, the bandwidth counter is reset.

Programming QoS Fixed Mode

Fixed mode is mode 0. To put the QoS generator in fixed mode, set the registers as shown in the following table.

Table 62. QoS Generator Register Values for Fixed Mode
Register	Field	Value
`I_main_QosGenerator_Mode`	`MODE`	0
`I_main_QosGenerator_Priority`	`P0`	Urgency for write transactions (default=0)
`I_main_QosGenerator_Priority`	`P1`	Urgency for read transactions (default=1)

Higher priority (urgency) values mean that a packet receives preferential treatment at each arbitration node.

Bandwidth and Saturation

Encoding Bandwidth Values

The QoS bandwidth setting is frequency-dependent. Low-frequency masters require a larger value to achieve the same bandwidth as higher frequency masters.

You can calculate the register value for a bandwidth as follows:

*I_main_QosGenerator_Bandwidth = ( bandwidth / frequency ) * 256

where bandwidth is in MBps and frequency is in MHz.

For example, to configure the FPGA-to-SDRAM 0 QoS generator for a bandwidth of 1000 MBps at 200 MHz, calculate the register value as follows:

fpga2sdram0_axi128_I_main_QosGenerator_Bandwidth = ( 1000 / 200 ) * 256 = 1280.

Encoding Saturation Values

The QoS saturation represents the period of time that the master bandwidth is evaluated. The saturation value represents units of 16 bytes of data transferred. For example a 64-bit master that should have the bandwidth re-evaluated every 100 clock cycles would use a saturation setting of 100 cycles * 8 bytes / 16 bytes =50

The saturation register controls the number of bytes that the initiator can transmit at full speed before the limiter or regulator takes effect. The register encodes the byte count as a multiple of 16 bytes. Therefore you can calculate the register value as follows:

*I_main_QosGenerator_Saturation = nbytes / 16

For example, to let the MPU initiator send 64 bytes at full speed, calculate the register value as follows:

mpu_m1_I_main_QosGenerator_Saturation = 64 / 16 = 4

QoS Generator Registers

There is a register set for each QoS generator.

Each QoS generator's register set includes the register types shown in the following table.

Table 63. QoS Generator Register Types
Name	Purpose
`I_main_QosGenerator_Priority`	Specifies master urgencies. Register usage depends on the QoS generator mode.
`I_main_QosGenerator_Mode`	Specifies the QoS generator mode (limiter, regulator, fixed, or bypass).
`I_main_QosGenerator_Bandwidth`	Specifies the bandwidth threshold for limiter and regulator modes.
`I_main_QosGenerator_Saturation`	Specifies the bandwidth measurement time window for limiter and regulator modes. ¹⁷

The actual register names consist of the register type plus a prefix specific to the QoS generator. For example, the registers for MPU subsystem L2 cache master 0 are:

mpu_m0_I_main_QosGenerator_Priority
mpu_m0_I_main_QosGenerator_Mode
mpu_m0_I_main_QosGenerator_Bandwidth
mpu_m0_I_main_QosGenerator_Saturation
mpu_m0_I_main_QosGenerator_ExtControl

The following table lists the prefixes for all QoS generators.

Table 64. QoS Register Name Prefixes
Initiator	Register Name Prefix
MPU Subsystem L2 Cache Master 0	`mpu_m0_`
MPU Subsystem L2 Cache Master 1	`mpu_m1_`
FPGA-to-HPS Bridge	`fpga2soc_axi32_`
FPGA-to-HPS Bridge	`fpga2soc_axi64_`
FPGA-to-HPS Bridge	`fpga2soc_axi128_`
DMA Controller	`dma_m0_`
Ethernet MAC 0	`emac0_m_`
Ethernet MAC 1	`emac1_m_`
Ethernet MAC 2	`emac2_m_`
USB-2 OTG 0	`usb0_m_`
USB-2 OTG 1	`usb1_m_`
NAND Flash Controller	`nand_m_`
SD/MMC Controller	`sdmmc_m_`
FPGA-to-SDRAM Port 0	`fpga2sdram0_axi32_`
FPGA-to-SDRAM Port 0	`fpga2sdram0_axi64_`
FPGA-to-SDRAM Port 0	`fpga2sdram0_axi128_`
FPGA-to-SDRAM Port 1	`fpga2sdram1_axi32_`
FPGA-to-SDRAM Port 1	`fpga2sdram1_axi64_`
FPGA-to-SDRAM Port 2	`fpga2sdram2_axi32_`
FPGA-to-SDRAM Port 2	`fpga2sdram2_axi64_`
FPGA-to-SDRAM Port 2	`fpga2sdram2_axi128_`

QoS Programming Examples

Example: One Master Always Takes Precedence

In this example, one particular master's packets always take precedence.

Consider a system that includes three masters, A, B, and C. In this system, we require that master A always takes precedence over master B at each arbitration node, and master B always takes precedence over master C. To implement this arbitration scheme, we configure all QoS generators in fixed mode, and assign appropriate values for read and write urgency, as shown in the following table:

Table 65. Fixed Example Settings
Master	QoS Mode	P1 (Read Urgency)	P0 (Write Urgency)
A	Fixed (mode 0)	`0x3`	`0x3`
B	Fixed (mode 0)	`0x2`	`0x2`
C	Fixed (mode 0)	`0x1`	`0x1`

In fixed mode, masters by default have a read urgency of 1 and write urgency of 0. So master C has equal urgency for reads and higher urgency for writes compared to all the other masters in the system.

Example: Tuning for Specific Throughput Requirements

In this example, we require each master to meet specific throughput requirements.

Consider a system in which masters A, B, and C have the following throughput requirements:

Master A: 1000 MBps operating at 100 MHz
Master B: 800 MBps operating at 800 MHz
Master C: 500 MBps operating at 400 Mhz

We achieve this by using the QoS generators in regulator mode. There are other masters in the system, and these three masters may need to "steal" bandwidth from them to achieve the required throughputs. Therefore, all the other QoS generators are configured in limiter mode to cap their bandwidth usage.

To set up the QoS generators for masters A, B, and C, we can use the settings in the following table.

Table 66. Regulator Example Settings
Master	QoS Mode	QoS Bandwidth	QoS Saturation (register value) ¹⁸
A	Regulator	2560	64 bytes (saturation = 4)
B	Regulator	256	256 bytes (saturation = 16)
C	Regulator	320	128 bytes (saturation = 8)

Not shown are the master urgencies. When a QoS regulator achieves the required bandwidth the urgency is downgraded to allow other masters to achieve their required bandwidth.

We also set the urgency threshold values, P1 and P0. P1 is the urgency when a master falls below the bandwidth threshold. P0 is the urgency when a master is above the bandwidth threshold. regulator mode ensures a minimum bandwidth to the master, by raising the urgency when the bandwidth falls below the threshold, and downgrading it when the bandwidth is back above the threshold.

For any master the value of P1 must be greater or equal than P0.

System Interconnect Address Map and Register Definitions

For complete HPS address map and register definitions, refer to the Intel Arria 10 HPS Register Address Map and Definitions.

⁸ The LWFPGASLAVES region is part of the "PERIPH" region.

⁹ For details about the noc_addr_remap_value register, refer to "Memory Map Remap Bits"

¹⁰ For details about the noc_addr_remap_value register, refer to "Memory Map Remap Bits".

¹¹ Security control register (SCR)

¹² Ensure that Avalon-MM burst transactions into the HPS do not cross the 4 KB address boundary restriction specified by the AXI* protocol.

¹³ Acceptance is the maximum number of transactions accepted.

¹⁴ "Boot Secure" means the slave is in the secure state after reset.

¹⁵ The Advanced Peripheral Bus ( APB* ) slave does not support byte enables. All writes must be 32 bits wide.

¹⁶ "Secure" means that the slave is permanently in the secure state.

¹⁷ For detailed information about setting bandwidth and saturation, refer to "Bandwidth and Saturation".

¹⁸ The QoS bandwidth register value is computed as (desired bandwidth (MBps) * 256) / master frequency (MHz)

Using Intel.com Search

Quick Links

Recent Searches

Advanced Search

Only search in

Intel® Arria® 10 Hard Processor System Technical Reference Manual

8. System Interconnect

About the System Interconnect

Features of the System Interconnect

System Interconnect Block Diagram and System Integration

System Interconnect Block Diagram

Connectivity

Arria 10 HPS Master-to-Slave Connectivity Matrix

On-Chip RAM

Peripherals Connections

System Connections

HPS-to-FPGA Bridge

SDRAM Connections

System Interconnect Architecture

SDRAM L3 Interconnect Architecture

Arria 10 HPS Secure Firewalls

About the Rate Adapters

About the SDRAM L3 Interconnect

Features of the SDRAM L3 Interconnect

SDRAM L3 Interconnect Block Diagram and System Integration

About the SDRAM Scheduler

About Arbitration and Quality of Service

About the Service Network

About the Observation Network

Functional Description of the System Interconnect

System Interconnect Address Spaces

Arria 10 HPS Available Address Maps

Notes on Address Maps

Arria 10 HPS L3 Address Space

SDRAM Window Region

On-Chip RAM Region, Low Mapping

HPS-to-FPGA Slaves Region

Lightweight HPS-to-FPGA Slaves Region

Peripherals Region

On-Chip RAM Region, High Mapping

MPU Address Space

Boot Region

SDRAM Window Region

HPS-to-FPGA Slaves Region

Lightweight HPS-to-FPGA Slaves Region

Peripherals Region

On-Chip RAM Region

Boot ROM Region

SCU and L2 Registers Region

SDRAM Address Space

Address Remapping

Memory Map Remap Bits

Secure Transaction Protection

System Interconnect Master Properties

Master Caching and Buffering Overrides

System Interconnect Slave Properties

System Interconnect Clocks

List of Main L3 Interconnect Clocks

List of SDRAM L3 Interconnect Clocks

System Interconnect Resets

Functional Description of the Rate Adapters

Functional Description of the Firewalls

Security

Slave Security

Master Security

Functional Description of the SDRAM L3 Interconnect

Functional Description of the SDRAM Scheduler

Monitors for Mutual Exclusion

Exclusive Access Support

Arbitration and Quality of Service in the SDRAM Scheduler

Functional Description of the SDRAM Adapter

ECC

ECC Write Behavior

ECC Read Behavior

SDRAM Adapter Interrupt Support

SDRAM Adapter Clocks

SDRAM L3 Firewalls

SDRAM L3 Interconnect Resets

Functional Description of the Arbitration Logic

Functional Description of the QoS Generators