Functional Description of the 8K DisplayPort Video Format Conversion Design Example
The Platform Designer system, udx10_dp.qsys, contains the DisplayPort receiver and transmitter protocol IP, the video pipeline IP, and the Nios II processor components. The design connects the Platform Designer system to the DisplayPort receiver and transmitter PHY logic (which contains the interface transceivers) and the transceiver reconfiguration logic at the top level in a Verilog HDL RTL design file (top.v). The design comprises a single video processing path between the DisplayPort input and the DisplayPort output.
Figure 2. Block Diagram
The diagram shows the blocks in the 8K DisplayPort Video Format Conversion Design Example. The diagram does not show some of the generic peripherals connected to the Nios II, the Avalon-MM between the Nios II processor, and the other components of the system. The design accepts video from a DisplayPort source on the left, processes the video through the video pipeline from left to right before passing the video out to the DisplayPort sink on the right.
DisplayPort Receiver PHY and DisplayPort Receiver IP
The Bitec DisplayPort FMC card provides a buffer for the DisplayPort 1.4 signal from the DisplayPort source. The combination of DisplayPort Receiver PHY and DisplayPort Receiver IP decodes the incoming signal to create a video stream. The DisplayPort receiver PHY contains the transceivers to deserialize the incoming data and the DisplayPort receiver IP decodes the DisplayPort protocol. The combined DisplayPort Receiver IP processes the incoming DisplayPort signal without any software. The resulting video signal from the DisplayPort receiver IP is a native packetized streaming format. The design configures the DisplayPort receiver for 10-bit output.
DisplayPort to Clocked Video IP
The packetized streaming data format output by the DisplayPort receiver is not directly compatible with the clocked video data format that the Clocked Video Input IP expects. The DisplayPort to Clocked Video IP is a custom IP for this design. It converts the DisplayPort output into a compatible clocked video format that you can connect directly to the Clocked Video Input. The DisplayPort to Clocked Video IP can modify the wire signaling standard and can alter the ordering of the color planes within each pixel. The DisplayPort standard specifies color ordering that is different than the Intel video pipeline IP ordering. The Nios II processor controls the color swap. It reads the current color space for transmission from the DisplayPort receiver IP with its Avalon-MM slave interface. It directs the DisplayPort to Clocked Video IP to apply the appropriate correction with its Avalon-MM slave interface.
Clocked Video Input
The clocked video input processes the clocked video interface signal from the DisplayPort to Clocked Video IP and converts it to Avalon-ST Video signal format. This signal format strips all horizontal and vertical blanking information from the video leaving only active picture data. The IP packetizes it as one packet per video frame. It also adds additional metadata packets (referred to as control packets) that describe the resolution of each video frame. The Avalon-ST Video stream through the processing pipe is four pixels in parallel, with three symbols per pixel. The clocked video input provides clock crossing for the conversion from the variable rate clocked video signal from the DisplayPort receiver IP to the fixed clock rate (300 MHz) for the video IP pipeline.
Stream Cleaner
The stream cleaner ensures that the Avalon-ST Video signal passing to the processing pipeline is error free. Hot plugging of the DisplayPort source can cause the design to present incomplete frames of data to the clocked video input IP and to generate errors in the resulting Avalon-ST Video stream. The size of the packets containing the video data for each frame then do not match the size reported by the associated control packets. The stream cleaner detects these conditions and adds additional data (grey pixels) to the end of the offending video packets to complete the frame and match the specification in the control packet.
Chroma Resampler (Input)
The video data that the design receives at the input from DisplayPort may be 4:4:4, 4:2:2, or 4:2:0 chroma sampled. The input chroma resampler takes the incoming video in any format and converts it to 4:4:4 in all cases. To provide higher visual quality, the chroma resampler uses the most computationally expensive filtered algorithm. The Nios II processor reads the current chroma sampling format from the DisplayPort receiver IP via its Avalon-MM slave interface. It communicates the format to the chroma resampler via its Avalon-MM slave interface.
Color Space Converter (Input)
The input video data from DisplayPort may use either the RGB or YCbCr color space. The input color space converter takes the incoming video in whatever format it arrives and converts it to RGB in all cases. The Nios II processor reads the current color space from the DisplayPort receiver IP with its Avalon-MM slave interface; it loads the correct conversion coefficients to the chroma resampler through its Avalon-MM slave interface.
Clipper
The clipper selects an active area from the incoming video stream and discards the remainder. The software control running on the Nios II processor defines the region to select. The region depends on the resolution of the data received at the DisplayPort source and the output resolution and scaling mode. The processor communicates the region to the Clipper through its Avalon-MM slave interface.
Scaler
The design applies scaling to the incoming video data according to the input resolution received, and the output resolution you require. You may also select between three scaling modes (upscale, downscale and passthrough). Two Scalar IPs provide the scaling functionality: one implements any required downscaling; the other implements upscaling. The design requires two scalers.
- When the scaler implements a downscale, it does not produce valid data on every clock cycle at its output. For example, if implementing a 2x downscale ratio, the valid signal at the output is high every other clock cycle while the design receives each even numbered input line, and then low for the entirety of the odd numbered input lines. This bursting behavior is fundamental to the process of reducing the data rate at the output, but is incompatible with the downstream Mixer IP, which generally expects a more consistent data rate to avoid underflow at the output. The design requires the Frame Buffer between any downscale and mixer. The Frame Buffer allows the Mixer to read the data at the rate it requires.
- When the scaler implements an upscale, it produces valid data on every clock cycle, so the following mixer has no issues. However, it may not accept new input data on every clock cycle. Taking a 2x upscale as an example, on the even numbered output lines it accepts a new beat of data every other clock cycle, then accepts no new input data on the odd numbered output lines. However, the upstream Clipper may produce data at an entirely different rate if it is applying a significant clip (e.g. during a zoom-in). Therefore, a Clipper and upscale must generally be separated by a Frame Buffer, requiring the Scaler to sit after the Frame Buffer in the pipeline. The Scaler must sit before the Frame Buffer for downscales, so the design implements two separate scalers either side of the Frame Buffer: one for upscale; the other for downscale.
Two Scalers also reduce the maximum DDR4 bandwidth required by the Frame Buffer. You must always apply downscales before the Frame Buffer, minimizing the data rate on the write side. Always apply upscales after the Frame Buffer, which minimizes the data rate on the read side.
Each Scaler gets the required input resolution from the control packets in the incoming video stream, while the Nios II processor with the Avalon-MM slave interface sets the output resolution for each Scaler.
Frame Buffer
The frame buffer uses the DDR4 memory to perform triple buffering that allows the video and image processing pipeline to perform frame rate conversion between the incoming and outgoing frame rates. The design can accept any input frame rate, but the total pixel rate must not exceed 1 giga pixels per second. The Nios II software sets the output frame rate to either 30 or 60 fps, according to the output mode you select. The output frame rate is a function of the Clocked Video Output settings and the output video pixel clock. The backpressure that the Clocked Video Output applies to the pipeline determines the rate at which the read side of the Frame Buffer pulls video frames from the DDR4.
Mixer
The mixer generates a fixed size black background image that the Nios II processor programs to match the size of the current output image. The mixer has two inputs. The first input connects to the upscaler to allow the design to show the output from the current video pipeline. The second input connects to the icon generator block. The design only enables the mixer's first input when it detects active, stable video at the clocked video input. Therefore, the design maintains a stable output image at the output while hot-plugging at the input. The design alpha blends the second input to the mixer, connected to the icon generator, over both the background and video pipeline images with 50% transparency.
Color Space Converter (Output)
The output color space converter transforms the input RGB video data to either RGB or YCbCr color space based on the runtime setting from software.
Chroma Resampler (Output)
The output chroma resampler converts the format from 4:4:4 to one of 4:4:4, 4:2:2, or 4:2:0 formats. The software sets the format. The output chroma resampler also uses filtered algorithm to achieve high-quality video.
Clocked Video Output
The clocked video output converts the Avalon-ST Video stream to the clocked video format. The clocked video output adds horizontal and vertical blanking and synchronization timing information to the video. The Nios II processor programs the relevant settings in the clocked video output depending on the output resolution and frame rate that you request. The clocked video output converts the clock, crossing from the fixed 300 MHz pipeline clock to the variable rate of the clocked video.
Clocked Video to DisplayPort
The DisplayPort transmitter component accepts data formatted as clocked video. Differences in the wire signaling and declaration of the conduit interfaces in Platform Designer prevent you connecting the Clocked Video Output directly to the DisplayPort transmitter IP. The Clocked Video to DisplayPort component is design-specific custom IP to provide the simple conversion required between the Clocked Video Output and the DisplayPort transmitter IP. It also swaps the ordering of the color planes in each pixel to account for the different color formatting standards used by Avalon-ST Video and DisplayPort.
DisplayPort Transmitter IP and DisplayPort Transmitter PHY
The DisplayPort transmitter IP and DisplayPort transmitter PHY together work to convert the video stream from clocked video to a compliant DisplayPort stream. The DisplayPort transmitter IP handles the DisplayPort protocol and encodes the valid DisplayPort data, while the DisplayPort transmitter PHY contains the transceivers and creates the high-speed serial output.
Nios II Processor and Peripherals
The Platform Designer system contains a Nios II processor, which manages the DisplayPort receiver and transmitter IPs and the runtime settings for the processing pipeline. The Nios II processor connects to these basic peripherals:
- An on-chip memory to store the program and its data.
- A JTAG UART to display software printf output (via a Nios II terminal).
- A system timer to generate millisecond level delays at various points in the software, as required by the DisplayPort specification of minimum event durations.
- LEDs to display system status.
- Push-button switches to allow switching between scaling modes and to enable and disable display of the Intel logo.
- DIP switches to allow switching of the output format and to enable and disable the printing of messages to a Nios II terminal.
Hot-plug events on both the DisplayPort source and sink fire interrupts that trigger the Nios II Processor to configure the DisplayPort transmitter and pipeline correctly. The main loop in the software code also monitors that values on the push-buttons and DIP switches and alters the pipeline setup accordingly.
I²C Controllers
The design contains two I²C controllers (Si5338 and PS8460) to edit the settings of three of the other components on the Intel® Arria® 10 10 GX FPGA Development Kit.
Two Si5338 clock generators on the Intel® Arria® 10 GX FPGA Development Kit connect to the same I²C bus. The first generates the reference clock for the DDR4 EMIF. By default, this clock is set to 100 MHz for use with 1066 MHz DDR4, but this design runs the DDR4 at 1200 MHz, which requires a reference clock of 150 MHz. At startup the Nios II processor, via the I²C controller peripheral, changes the settings in the register map of the first Si5338 to increase the speed of the DDR4 reference clock to 150MHz. The second Si5338 clock generator generates the vid_clk for the clocked video interface between the pipeline and the DisplayPort transmitter IP. You must adjust the speed of this clock for each different output resolution and frame rate supported by the design. You can adjust the speed at run time when the Nios II processor requires.
The Bitec DisplayPort 1.4 FMC daughter card makes use of the Parade PS8460 jitter cleaning repeater and retimer. At startup the Nios II processor edits the default settings of this component to meet the requirements of the design.