Video and Image Processing Suite User Guide

Download PDF
ID 683416
Date 4/04/2022
Public
Document Table of Contents
Document Table of Contents x
1. About the Video and Image Processing Suite 2. Avalon Streaming Video 3. Clocked Video 4. VIP Run-Time Control 5. Getting Started 6. VIP Connectivity Interfacing 7. Clocked Video Interface IPs 8. 2D FIR II IP Core 9. Mixer II IP Core 10. Clipper II IP Core 11. Color Plane Sequencer II IP Core 12. Color Space Converter II IP Core 13. Chroma Resampler II IP Core 14. Control Synchronizer IP Core 15. Deinterlacer II IP Core 16. Frame Buffer II IP Core 17. Gamma Corrector II IP Core 18. Configurable Guard Bands IP Core 19. Interlacer II IP Core 20. Scaler II IP Core 21. Switch II IP Core 22. Test Pattern Generator II IP Core 23. Trace System IP Core 24. Warp Lite Intel FPGA IP 25. Avalon-ST Video Stream Cleaner IP Core 26. Avalon-ST Video Monitor IP Core 27. VIP IP Core Software Control 28. Security Considerations 29. Video and Image Processing Suite User Guide Archives 30. Document Revision History for the Video and Image Processing Suite User Guide A. Avalon-ST Video Verification IP Suite
1. About the Video and Image Processing Suite x
1.1. Release Information 1.2. Device Family Support 1.3. Latency 1.4. In-System Performance and Resource Guidance 1.5. Stall Behavior and Error Recovery
2. Avalon Streaming Video x
2.1. Avalon-ST Video Configuration Types 2.2. Avalon-ST Video Packet Types 2.3. Avalon-ST Video Operation 2.4. Avalon-ST Video Error Cases
2.2. Avalon-ST Video Packet Types x
2.2.1. Avalon-ST Video Control Packets 2.2.2. Avalon-ST Video Video Packets 2.2.3. Avalon-ST Video User Packets
3. Clocked Video x
3.1. Video Formats
3.1. Video Formats x
3.1.1. Embedded Synchronization Format: Clocked Video Output 3.1.2. Embedded Synchronization Format: Clocked Video Input 3.1.3. Separate Synchronization Format 3.1.4. Video Locked Signal 3.1.5. Clocked Video and 4:2:0 Chroma Subsampling
3.1.1. Embedded Synchronization Format: Clocked Video Output x
3.1.1.1. Clocked Video Output and SDI II TX Interface
3.1.5. Clocked Video and 4:2:0 Chroma Subsampling x
3.1.5.1. Avalon-ST Video Control Packets for 4:2:0 Video 3.1.5.2. 4:2:0 Clocked Video 3.1.5.3. Resampling 4:2:0
5. Getting Started x
5.1. IP Catalog and VIP Parameter Editor 5.2. Installing and Licensing IP Cores
5.1. IP Catalog and VIP Parameter Editor x
5.1.1. Specifying IP Core Parameters and Options
5.2. Installing and Licensing IP Cores x
5.2.1. Intel® FPGA IP Evaluation Mode
6. VIP Connectivity Interfacing x
6.1. Avalon-ST Color Space Mappings
6.1. Avalon-ST Color Space Mappings x
6.1.1. Interfacing with High-Definition Multimedia Interface (HDMI) 6.1.2. Interfacing with DisplayPort 6.1.3. Interfacing with Serial Digital Interface (SDI) 6.1.4. Unsupported SDI Mappings 6.1.5. 12G-SDI
6.1.5. 12G-SDI x
6.1.5.1. 12G-SDI with Clocked Video Input II 6.1.5.2. 12G-SDI with Clocked Video Output II
7. Clocked Video Interface IPs x
7.1. Supported Features for Clocked Video Output II IP 7.2. Control Port 7.3. Clocked Video Input IP Format Detection 7.4. Clocked Video Output IP Video Modes 7.5. Clocked Video Output II Latency Mode 7.6. Generator Lock 7.7. Underflow and Overflow 7.8. Timing Constraints 7.9. Handling Ancillary Packets 7.10. Modules for Clocked Video Input II IP Core 7.11. Clocked Video Input II Signals, Parameters, and Registers 7.12. Clocked Video Output II Signals, Parameters, and Registers
7.3. Clocked Video Input IP Format Detection x
7.3.1. Format Detection in Clocked Video Input II 7.3.2. Interrupts
7.4. Clocked Video Output IP Video Modes x
7.4.1. Interrupts
7.11. Clocked Video Input II Signals, Parameters, and Registers x
7.11.1. Clocked Video Input II Interface Signals 7.11.2. Clocked Video Input II Parameter Settings 7.11.3. Clocked Video Input II Control Registers
7.12. Clocked Video Output II Signals, Parameters, and Registers x
7.12.1. Clocked Video Output II Interface Signals 7.12.2. Clocked Video Output II Parameter Settings 7.12.3. Clocked Video Output II Control Registers
8. 2D FIR II IP Core x
8.1. 2D FIR Filter Processing 8.2. 2D FIR Filter Precision 8.3. 2D FIR Coefficient Specification 8.4. 2D FIR Filter Symmetry 8.5. Result to Output Data Type Conversion 8.6. Edge-Adaptive Sharpen Mode 8.7. 2D FIR Filter Parameter Settings 8.8. 2D FIR Filter Control Registers
8.4. 2D FIR Filter Symmetry x
8.4.1. No Symmetry 8.4.2. Horizontal Symmetry 8.4.3. Vertical Symmetry 8.4.4. Horizontal and Vertical Symmetry 8.4.5. Diagonal Symmetry
8.6. Edge-Adaptive Sharpen Mode x
8.6.1. Edge Detection 8.6.2. Filtering 8.6.3. Precision
9. Mixer II IP Core x
9.1. Alpha Blending 9.2. Mixer II Parameter Settings 9.3. Mixer II Control Registers 9.4. Layer Mapping 9.5. Low-Latency Mode
10. Clipper II IP Core x
10.1. Clipper II Parameter Settings 10.2. Clipper II Control Registers
11. Color Plane Sequencer II IP Core x
11.1. Combining Color Patterns 11.2. Rearranging Color Patterns 11.3. Splitting and Duplicating 11.4. Handling of Subsampled Data 11.5. Handling of Non-Image Avalon-ST Packets 11.6. Color Plane Sequencer Parameter Settings
12. Color Space Converter II IP Core x
12.1. Input and Output Data Types 12.2. Color Space Conversion 12.3. Result of Output Data Type Conversion 12.4. Color Space Converter Parameter Settings 12.5. Color Space Conversion Control Registers
12.2. Color Space Conversion x
12.2.1. Predefined Conversions
13. Chroma Resampler II IP Core x
13.1. Chroma Resampler Algorithms 13.2. Chroma Resampler Parameter Settings 13.3. Chroma Resampler Control Registers
13.1. Chroma Resampler Algorithms x
13.1.1. Nearest Neighbor 13.1.2. Bilinear 13.1.3. Filtered
14. Control Synchronizer IP Core x
14.1. Using the Control Synchronizer IP Core 14.2. Control Synchronizer Parameter Settings 14.3. Control Synchronizer Control Registers
15. Deinterlacer II IP Core x
15.1. Deinterlacing Algorithm Options 15.2. Deinterlacing Algorithms 15.3. Run-time Control 15.4. Pass-Through Mode for Progressive Frames 15.5. Cadence Detection (Motion Adaptive Deinterlacing Only) 15.6. Avalon-MM Interface to Memory 15.7. Motion Adaptive Mode Bandwidth Requirements 15.8. Avalon-ST Video Support 15.9. 4K Video Passthrough Support 15.10. Behavior When Unexpected Fields are Received 15.11. Handling of Avalon-ST Video Control Packets 15.12. Deinterlacer II Parameter Settings 15.13. Deinterlacing Control Registers
15.2. Deinterlacing Algorithms x
15.2.1. Vertical Interpolation (Bob) 15.2.2. Field Weaving (Weave) 15.2.3. Motion Adaptive 15.2.4. Motion Adaptive High Quality (Sobel Edge Interpolation)
15.9. 4K Video Passthrough Support x
15.9.1. Approach A 15.9.2. Approach B
15.13. Deinterlacing Control Registers x
15.13.1. Scene Change Motion Multiplier Value 15.13.2. Tuning Motion Shift and Motion Scale Registers
16. Frame Buffer II IP Core x
16.1. Double Buffering 16.2. Triple Buffering 16.3. Locked Frame Rate Conversion 16.4. Handling of Avalon-ST Video Control Packets and User Packets 16.5. Frame Buffer Parameter Settings 16.6. Frame Buffer Application Examples 16.7. Frame Buffer Control Registers
16.3. Locked Frame Rate Conversion x
16.3.1. Converting Frame Rates
16.7. Frame Buffer Control Registers x
16.7.1. Frame Writer Only Mode 16.7.2. Frame Reader Only Mode 16.7.3. Memory Map for Frame Reader or Writer Configurations
17. Gamma Corrector II IP Core x
17.1. Gamma Corrector Parameter Settings 17.2. Gamma Corrector Control Registers
18. Configurable Guard Bands IP Core x
18.1. Guard Bands Parameter Settings 18.2. Configurable Guard Bands Control Registers
19. Interlacer II IP Core x
19.1. Interlacer Parameter Settings 19.2. Interlacer Control Registers
20. Scaler II IP Core x
20.1. Nearest Neighbor Algorithm 20.2. Bilinear Algorithm 20.3. Polyphase and Bicubic Algorithm 20.4. Edge-Adaptive Scaling Algorithm 20.5. Scaler II Parameter Settings 20.6. Scaler II Control Registers
20.2. Bilinear Algorithm x
20.2.1. Bilinear Algorithmic Description
20.3. Polyphase and Bicubic Algorithm x
20.3.1. Double-Buffering 20.3.2. Polyphase Algorithmic Description 20.3.3. Choosing and Loading Coefficients
21. Switch II IP Core x
21.1. Switch II Parameter Settings 21.2. Switch II Control Registers
22. Test Pattern Generator II IP Core x
22.1. Test Patterns 22.2. Output Subsampling and Color Space 22.3. Generation of Avalon-ST Video Control Packets and Run-Time Control 22.4. Test Pattern Generator II Parameter Settings 22.5. Test Pattern Generator II Control Registers
22.1. Test Patterns x
22.1.1. Color Bars 22.1.2. Grayscale Bars 22.1.3. Black and White Bars 22.1.4. SDI Pathological 22.1.5. Uniform Background
23. Trace System IP Core x
23.1. Trace System Parameter Settings 23.2. Trace System Signals 23.3. Operating the Trace System from System Console
23.3. Operating the Trace System from System Console x
23.3.1. Loading the Project and Connecting to the Hardware 23.3.2. Trace Within System Console 23.3.3. TCL Shell Commands
24. Warp Lite Intel FPGA IP x
24.1. Warp Lite IP Release Information 24.2. About Image Warping 24.3. About the Warp Lite Intel FPGA IP 24.4. Operating the Warp Lite IP 24.5. Warp Lite IP Parameters 24.6. Warp Lite IP Control Registers' Map 24.7. Warp Lite IP Line Store 24.8. Warp Lite IP API
24.3. About the Warp Lite Intel FPGA IP x
24.3.1. Warp Definition Equations 24.3.2. Resampling and Interpolation
24.3.1. Warp Definition Equations x
24.3.1.1. Transform Matrix and Keystone Region Corners 24.3.1.2. Horizontal Flip Equation
24.7. Warp Lite IP Line Store x
24.7.1. Line Store RAM Utilization 24.7.2. Buffer Ingress and Egress Overlap
25. Avalon-ST Video Stream Cleaner IP Core x
25.1. Avalon-ST Video Protocol 25.2. Repairing Non-Ideal and Error Cases 25.3. Avalon-ST Video Stream Cleaner Parameter Settings 25.4. Avalon-ST Video Stream Cleaner Control Registers
26. Avalon-ST Video Monitor IP Core x
26.1. Packet Visualization 26.2. Monitor Settings 26.3. Avalon-ST Video Monitor Parameter Settings 26.4. Avalon-ST Video Monitor Control Registers
27. VIP IP Core Software Control x
27.1. HAL Device Drivers for Nios II SBT
28. Security Considerations x
28.1. Using Avalon-ST Video Monitor Debug Tools 28.2. Configuring Memory Subsystems
A. Avalon-ST Video Verification IP Suite x
A.1. Avalon-ST Video Class Library A.2. Example Tests A.3. Complete Class Reference A.4. Data Format
A.2. Example Tests x
A.2.1. Generating the Testbench Netlist A.2.2. Running the Test in Intel® Quartus® Prime Standard Edition A.2.3. Running the Test in Intel® Quartus® Prime Pro Edition A.2.4. Viewing the Video File A.2.5. Verification Files A.2.6. Constrained Random Test
A.2.5. Verification Files x
A.2.5.1. bfm_drivers.sv A.2.5.2. nios_control_model.sv
A.3. Complete Class Reference x
A.3.1. c_av_st_video_control A.3.2. c_av_st_video_data A.3.3. c_av_st_video_file_io A.3.4. c_av_st_video_item A.3.5. c_av_st_video_source_sink_base A.3.6. c_av_st_video_sink_bfm_’SINK A.3.7. c_av_st_video_source_bfm_’SOURCE A.3.8. c_av_st_video_user_packet A.3.9. c_pixel A.3.10. av_mm_transaction A.3.11. av_mm_master_bfm_`MASTER_NAME A.3.12. av_mm_slave_bfm_`SLAVE_NAME A.3.13. av_mm_control_register A.3.14. av_mm_control_base
1. About the Video and Image Processing Suite
2. Avalon Streaming Video
3. Clocked Video
4. VIP Run-Time Control
5. Getting Started
6. VIP Connectivity Interfacing
7. Clocked Video Interface IPs
8. 2D FIR II IP Core
9. Mixer II IP Core
10. Clipper II IP Core
11. Color Plane Sequencer II IP Core
12. Color Space Converter II IP Core
13. Chroma Resampler II IP Core
14. Control Synchronizer IP Core
15. Deinterlacer II IP Core
16. Frame Buffer II IP Core
17. Gamma Corrector II IP Core
18. Configurable Guard Bands IP Core
19. Interlacer II IP Core
20. Scaler II IP Core
21. Switch II IP Core
22. Test Pattern Generator II IP Core
23. Trace System IP Core
24. Warp Lite Intel FPGA IP
25. Avalon-ST Video Stream Cleaner IP Core
26. Avalon-ST Video Monitor IP Core
27. VIP IP Core Software Control
28. Security Considerations
29. Video and Image Processing Suite User Guide Archives
30. Document Revision History for the Video and Image Processing Suite User Guide
A. Avalon-ST Video Verification IP Suite

A.3.3. c_av_st_video_file_io

The declaration for the c_av_st_video_file_io class: 
class c_av_st_video_file_io#(parameter BITS_PER_CHANNEL = 8,
CHANNELS_PER_PIXEL = 3);
Table 95.  Method Calls for c_av_st_video_file_io Class
Method Call Description
function void set_send_control_packets(t_packet_controls); If this method is used to set the send_control_packet control to off, then one control packet is sent at the beginning of video data, but no further control packets are sent.
function t_packet_control get_send_control_packets(); —
function void set_send_user_packets(t_packet_control s); If the send_user_packets control is off, no user packets at all are sent. Otherwise, user packets are sent before and after any control packets.
function t_packet_control get_send_user_packets(); —
function void set_send_early_eop_packets(t_packet_control s); If the send_eop_packets control is off, all packets are of the correct length (or longer). Otherwise, early EOP are sent of a length determined by the constraints on early_eop_packet_length.
function t_packet_control get_send_early_eop_packets(); —
function void set_early_eop_probability(int s); If the send_early_eop_packets control is set to random, the early_eop_probability control determines what proportion of video packets are terminated early.
function int get_early_eop_probability(); —
function void set_send_late_eop_packets(t_packet_controls); If the send_late_eop_packets control is off, all packets are of the correct length (or longer). Otherwise, late EOP are sent of a length determined by the constraints on late_eop_packet_length.
function t_packet_control get_send_late_eop_packets(); —
function void set_late_eop_probability (int s); If the send_late_eop_packets control is set to random, the late_eop_probability control determines what proportion of video packets are terminated late.
function int get_late_eop_probability (); —
function void set_user_packet_probability (int s); If the send_user_packets is set to random, the user_packet_probability control determines the probability that a user packet being sent before a control packet. It also determines the probability that a user packet will be sent after a control packet.
function int get_user_packet_probability (); —
function void set_control_packet_probability(int s); If the send_control_packets control is set to random, the control_packet_probability control determines the probability of a control packet being sent before a video packet.
function int get_control_packet_probability(); —
function void set_send_garbage_after_control_packets (t_packet_control s); When the send_control_packet() method puts a control packet into the m_video_item_out mailbox, the append_garbage member of the control packet object is set to the value of send_garbage_after_control_packets.
function t_packet_control get_send_garbage_after_control_packets(); —
function void set_object_name(string s); You can use object_name to name a given object instance of a class to ensure any reporting that the class generates is labeled with the originating object’s name.
function string get_object_name(); —
function string get_filename(); This returns the filename associated with the object, by the open_file call.
function void set_image_height(bit[15:0]height); —
function bit[15:0]get_image_height(); —
function void set_image_width(bit[15:0] width); —
function bit[15:] get_image_width(); —
function void set_video_data_type(string s);
Sets the fourcc[3] code associated with the raw video data. The following are the supported four character code (FOURCC) codes:
  • RGB32
  • IYU2
  • YUY2
  • Y410
  • A2R10GB10
  • Y210
function string get_video_data_type(); Returns the FOURCC code (for example, RGB32) being used for the raw video data.
function int get_video_packets_handled(); —
function new(mailbox #(c_av_st_video_item)m_vid_out); Constructor. The mailbox is used to pass all packets in and out of the file I/O object.
function void open_file(string fname, t_rwrw); Files are opened using this method. For example:

video_file_reader.open_file(‘’vip_car_0.bin”, read);

t_rw is an enumerated type with values read or write.

NB. The read fails if there is no associated .spc file, for example, vip_car_o.spc).

function void close_file(); For example, video_file_reader.close_file();
task read_file(); Read_file() optionally calls send_user_packet() and send_control_packet(), then calls read_video_packet().
task send_control_packet(); The control packet sent is derived from the image height, width, and interlace fields as provided by open_file().
task send_user_packet(); The user packet sent is always comprised of random data and had a maximum length hard-coded to 33 data items.
task_generate_spc_file(); When writing a file, this call creates the necessary associated .spc file.
task read_video_packet(); The main file reading method call. Binary data is read from the file and packed into pixel objects according to the settings of ycbr_pixel_order and endianism. Pixel objects are packed into a video data object, with some pixels optionally added or discarded if late/early EOP is being applied. When one complete field of video has been read (as determined by the height and width controls), the video_data object is put in the mailbox.
task wait_for_and_write_video_packet_to_file(); When called, this method waits for an object to be put in the mailbox (usually from a sink BFM). When a control or a user packet object arrives, this call is reported and ignored. When a video packet arrives, the video data is written to the open file in little endianism format.
Table 96.  Members of c_av_st_video_file_io Class
Member Description

local int video_packets_handled = 0;

video_packets_handled is added whenever a packet is read or written to or from the file.
local int control_packets_handled = 0; control_packets_handled is added whenever a control packet is put in the object’s mailbox.
local int user_packets_handled = 0; user_packets_handled is added whenever a user packet is put in the object's mailbox.
local reg[15:0] image_height; —
local reg[15:0] image_width; —
local reg[3:0] image_interlaced; —
string image_fourcc; —
local string object_name = "file_io"; —
local string filename; —
local string spc_filename; —
local int fourcc_channels_per_pixel; Set when the associate .spc file is read.
local int fourcc_bits_per_channel; Set when the associate .spc file is read.
local int fourcc_pixels_per_word; Set when the associate .spc file is read.
local int fourcc_channel_lsb; Set when the associate .spc file is read.
int early_eop_probability = 20; —
Int late_eop_probability = 20; —
int user_packet_probability = 20; —
int control_packet_probability = 20; —
mailbox #(c_av_st_video_item) m_video_item_out = new(0); The mailbox is used to pass all packets in/out of the file i/o object.
rand t_packet_control send_control_packets = on; —
rand t_packet_control send_user_packets = off; —
rand t_packet_control send_early_eop_packets = off; —
rand t_packet_control send_late_eop_packets = off; If both send_late_eop_packets and send_early_eop_packets are set to random, a late EOP will only be generated if an early EOP has not been generated.
rand t_packet_control send_garbage_after_control_packets = off; —
rand int early_eop_packet_length = 20; constraint early_eop_length { early_eop_packet_length dist {1:= 10, [2:image_height*image_width-1]:/90}; early_eop_packet_length inside {[1:image_height*image_width]}; }
rand int late_eop_packet_length = 20; constraint late_eop_length { late_eop_packet_length inside {[1:100]}; }
Level Two Title