Video and Vision Processing Suite Intel® FPGA IP User Guide

Download PDF
ID 683329
Date 2/15/2022
Public

A newer version of this document is available. Customers should click here to go to the newest version.

Document Table of Contents
Document Table of Contents x
1. About the Video and Vision Processing Suite 2. Getting Started with the Video and Vision Processing IPs 3. Video and Vision Processing IPs Functional Description 4. Video and Vision Processing IP Interfaces 5. Video and Vision Processing IP Registers 6. Video and Vision Processing IPs Software Programming Model 7. Protocol Converter Intel® FPGA IP 8. 3D LUT Intel® FPGA IP 9. Chroma Resampler Intel® FPGA IP 10. Clipper Intel® FPGA IP 11. Clocked Video to Full Raster Converter Intel® FPGA IP 12. Color Space Converter Intel® FPGA IP 13. Full Raster to Clocked Video Converter Intel FPGA IP 14. Full Raster to Streaming Converter Intel® FPGA IP 15. Guard Bands Intel® FPGA IP 16. Mixer Intel FPGA IP 17. Pixels in Parallel Converter IP 18. Scaler 19. Tone Mapping Operator Intel® FPGA IP 20. Test Pattern Generator Intel FPGA IP 21. Video Frame Buffer Intel FPGA IP 22. Video Streaming FIFO Intel FPGA IP 23. Warp Intel® FPGA IP 24. Document Revision History for Video and Vision Processing Suite User Guide
1. About the Video and Vision Processing Suite x
1.1. Device Family Support 1.2. Video and Vision Processing IPs Release Information 1.3. Video and Vision Processing IPs Features 1.4. Lite versus Full IP Variants 1.5. Glossary of Video and Vision Terminology
2. Getting Started with the Video and Vision Processing IPs x
2.1. Video and Vision Processing IP Time-out Behavior 2.2. Generating a Video and Vision Processing IP 2.3. Simulating the Video and Vision Processing IPs
3. Video and Vision Processing IPs Functional Description x
3.1. Auxiliary Control Packets 3.2. Latency 3.3. IP Debugging Features 3.4. Stall Behavior and Error Recovery 3.5. Avalon Streaming Video Protocol Versus Intel FPGA Streaming Video Protocol
5. Video and Vision Processing IP Registers x
5.1. Video and Vision Processing IP Control Examples
7. Protocol Converter Intel® FPGA IP x
7.1. About the Protocol Converter IP 7.2. Protocol Converter IP Parameters 7.3. Protocol Converter IP Functional Description 7.4. Protocol Converter IP Interfaces 7.5. Protocol Converter IP Registers 7.6. Protocol Converter IP Software API
7.1. About the Protocol Converter IP x
7.1.1. Protocol Converter IP Performance and Resources
8. 3D LUT Intel® FPGA IP x
8.1. About the 3D LUT Intel® FPGA IP 8.2. 3D LUT IP Parameters 8.3. 3D LUT IP Block Description 8.4. 3D LUT IP Registers 8.5. 3D LUT IP Software API
8.1. About the 3D LUT Intel® FPGA IP x
8.1.1. 3D LUT IP Features 8.1.2. 3D LUT IP Performance IP and Resource Information
8.3. 3D LUT IP Block Description x
8.3.1. 3D LUT IP Interfaces 8.3.2. 3D LUT IP Latency
9. Chroma Resampler Intel® FPGA IP x
9.1. About the Chroma Resampler IP 9.2. Chroma Resampler IP Parameters 9.3. Chroma Resampler IP Functional Description 9.4. Chroma Resampler IP Registers 9.5. Chroma Resampler IP Software API
9.1. About the Chroma Resampler IP x
9.1.1. Chroma Resampler Performance and Resources
9.3. Chroma Resampler IP Functional Description x
9.3.1. Chroma Resampler IP Interfaces
10. Clipper Intel® FPGA IP x
10.1. About the Clipper IP 10.2. Clipper IP Parameters 10.3. Clipper IP Interfaces 10.4. Clipper IP Registers 10.5. Clipper IP Software API
10.1. About the Clipper IP x
10.1.1. Clipper IP Performance and Resources
11. Clocked Video to Full Raster Converter Intel® FPGA IP x
11.1. About the Clocked Video to Full Raster Converter IP 11.2. Clocked Video to Full Raster Converter Parameters 11.3. Clocked Video to Full Raster Converter Block Description 11.4. Clocked Video Converter to Full Raster IP Registers
11.1. About the Clocked Video to Full Raster Converter IP x
11.1.1. Clocked Video to Full Raster Converter Features 11.1.2. Clocked Video to Full Raster Converter Performance and Resources
11.3. Clocked Video to Full Raster Converter Block Description x
11.3.1. Clocked Video to Full Raster Converter Interfaces
12. Color Space Converter Intel® FPGA IP x
12.1. About the Color Space Converter IP 12.2. Color Space Converter IP Parameters 12.3. Color Space Converter IP Functional Description 12.4. Color Space Converter IP Registers 12.5. Color Space Converter IP Software API
12.1. About the Color Space Converter IP x
12.1.1. Color Space Converter IP Features 12.1.2. Color Space Converter IP Performance and Resources
12.3. Color Space Converter IP Functional Description x
12.3.1. Color Space Converter IP Interfaces
13. Full Raster to Clocked Video Converter Intel FPGA IP x
13.1. About the Full Raster to Clocked Video Converter 13.2. Full Raster to Clocked Video Converter Parameters 13.3. Full Raster to Clocked Video Converter Block Description 13.4. Full Raster to Clocked Video Converter Registers
13.1. About the Full Raster to Clocked Video Converter x
13.1.1. Full Raster to Clocked Video Converter Features 13.1.2. Full Raster to Clocked Video Converter Performance and Resource Utilization
13.3. Full Raster to Clocked Video Converter Block Description x
13.3.1. Full Raster to Clocked Video Converter Interfaces
14. Full Raster to Streaming Converter Intel® FPGA IP x
14.1. About the Full Raster to Streaming Converter IP 14.2. Full Raster to Streaming Converter Parameters 14.3. Full Raster to Streaming Converter Block Description
14.1. About the Full Raster to Streaming Converter IP x
14.1.1. Full Raster to Streaming Converter Features 14.1.2. Full Raster to Streaming Converter Performance and Resources
14.3. Full Raster to Streaming Converter Block Description x
14.3.1. Full Raster to Streaming Converter Interfaces
15. Guard Bands Intel® FPGA IP x
15.1. About the Guard Bands IP 15.2. Guard Bands IP Parameters 15.3. Guard Bands IP Interfaces 15.4. Guard Bands IP Registers 15.5. Guard Bands IP Software API
15.1. About the Guard Bands IP x
15.1.1. Guard Bands IP Performance and Resources
16. Mixer Intel FPGA IP x
16.1. About the Mixer IP 16.2. Mixer IP Parameters 16.3. Mixer IP Functional Description 16.4. Mixer IP Registers 16.5. Mixer IP Software API
16.1. About the Mixer IP x
16.1.1. Mixer IP Performance and Resources
16.3. Mixer IP Functional Description x
16.3.1. Mixer IP Interfaces
17. Pixels in Parallel Converter IP x
17.1. About the Pixels in Parallel Converter IP 17.2. Pixels in Parallel IP Converter Parameters 17.3. Pixels in Parallel Converter Interfaces 17.4. Pixels in Parallel Converter Registers 17.5. Pixels in Parallel Converter IP Software API
17.1. About the Pixels in Parallel Converter IP x
17.1.1. Pixels in Parallel IP Converter Performance and Resources
18. Scaler x
18.1. About the Scaler 18.2. Scaler IP Parameters 18.3. Scaler IP Functional Description 18.4. Scaler IP Registers 18.5. Scaler IP Software API
18.1. About the Scaler x
18.1.1. Scaler IP Performance and Resources
18.2. Scaler IP Parameters x
18.2.1. Updating Scaler IP Runtime Coefficients
18.3. Scaler IP Functional Description x
18.3.1. Scaler IP Interfaces
19. Tone Mapping Operator Intel® FPGA IP x
19.1. About the Tone Mapping Operator IP 19.2. TMO IP Parameters 19.3. TMO IP Block Description 19.4. TMO IP Registers 19.5. TMO IP Software API
19.1. About the Tone Mapping Operator IP x
19.1.1. TMO IP Features 19.1.2. TMO IP Performance and Resource Utilization
19.3. TMO IP Block Description x
19.3.1. TMO IP Interfaces 19.3.2. TMO IP Latency
20. Test Pattern Generator Intel FPGA IP x
20.1. About the Test Pattern Generator IP 20.2. Test Pattern Generator IP Parameters 20.3. Test Pattern Generator IP Functional Description 20.4. Test Pattern Generator IP Registers 20.5. Test Pattern Generator IP Software API
20.1. About the Test Pattern Generator IP x
20.1.1. Test Pattern Generator IP Performance and Resources
20.3. Test Pattern Generator IP Functional Description x
20.3.1. Test Pattern Generator IP Interfaces
21. Video Frame Buffer Intel FPGA IP x
21.1. About the Video Frame Buffer IP 21.2. Video Frame Buffer IP Parameters 21.3. Video Frame Buffer IP Functional Description 21.4. Video Frame Buffer IP Registers 21.5. Video Frame Buffer IP Software API
21.1. About the Video Frame Buffer IP x
21.1.1. Video Frame Buffer Performance and Resources
21.3. Video Frame Buffer IP Functional Description x
21.3.1. Video Frame Buffer IP Interfaces
22. Video Streaming FIFO Intel FPGA IP x
22.1. About the Video Streaming FIFO IP 22.2. Video Streaming FIFO IP Parameters 22.3. Video Streaming FIFO IP Interfaces
22.1. About the Video Streaming FIFO IP x
22.1.1. Video Streaming FIFO IP Performance and Resources
23. Warp Intel® FPGA IP x
23.1. About the Warp IP 23.2. Warp IP Parameters 23.3. Warp IP Block Description 23.4. Warp IP Registers 23.5. Warp IP Software API
23.1. About the Warp IP x
23.1.1. Warp IP Features 23.1.2. Warp IP Performance and Resource Utilization
23.3. Warp IP Block Description x
23.3.1. Warp IP Interfaces 23.3.2. Warp IP Latency 23.3.3. External Memory for Warp IP
23.5. Warp IP Software API x
23.5.1. Using Warp IP Software 23.5.2. Warp IP Software Code Examples
1. About the Video and Vision Processing Suite
2. Getting Started with the Video and Vision Processing IPs
3. Video and Vision Processing IPs Functional Description
4. Video and Vision Processing IP Interfaces
5. Video and Vision Processing IP Registers
6. Video and Vision Processing IPs Software Programming Model
7. Protocol Converter Intel® FPGA IP
8. 3D LUT Intel® FPGA IP
9. Chroma Resampler Intel® FPGA IP
10. Clipper Intel® FPGA IP
11. Clocked Video to Full Raster Converter Intel® FPGA IP
12. Color Space Converter Intel® FPGA IP
13. Full Raster to Clocked Video Converter Intel FPGA IP
14. Full Raster to Streaming Converter Intel® FPGA IP
15. Guard Bands Intel® FPGA IP
16. Mixer Intel FPGA IP
17. Pixels in Parallel Converter IP
18. Scaler
19. Tone Mapping Operator Intel® FPGA IP
20. Test Pattern Generator Intel FPGA IP
21. Video Frame Buffer Intel FPGA IP
22. Video Streaming FIFO Intel FPGA IP
23. Warp Intel® FPGA IP
24. Document Revision History for Video and Vision Processing Suite User Guide

23.4. Warp IP Registers

As the software API allows you to program and control the warp IP, it only has a limited set of registers.
Table 375.  Warp IP RegistersThese registers are all read-only and allow interrogation of the Warp IP’s parameter settings. All the registers are 32-bit wide.
Register Name Offset Address Access Type Description
vid_pid 0x000 RO Warp IP product and vendor ID
version_number 0x004 RO The version for this release of the Warp IP
Reserved 0x008 RO
Reserved 0x00C RO
pip 0x010 RO Indicates the value of pixels in parallel parameter
color_planes 0x014 RO Indicates the value of number of color planes parameter
cps 0x018 RO Indicates the value of bits per color sample parameter
num_engines 0x01C RO Indicates the value of number of engines parameter
max_input_width 0x020 RO Indicates the value of maximum input video width parameter
max_output_width 0x024 RO Indicates the value of maximum output video width parameter
memory_buffer_size 0x028 RO Indicates the value of memory frame buffer size parameter
easy_warp 0x02C RO Indicates the value of Use easy warp parameter
Reserved 0x030-0x16C RO -
int_control 0x170 RW Turns on the interrupt
int_status 0x174 RW1C Read interrupt status and clear interrupt
Table 376.   vid_pid Register
Bits Name Description
31:16 VID Vendor ID that returns a value of 0x6AF7
15:0 PID Warp product ID that returns a value of 0x016F
Table 377.   version_number Register
Bits Name Description
31:0 Version Number The version number of the Warp IP
Table 378.   pip Register
Bits Name Description
31:0 Pixels in Parallel The pixel in parallel parameter. Returns a value of 1 or 2.
Table 379.   color_planes Register
Bits Name Description
31:0 Number of Color Planes The number of color planes parameter. Returns a value of 3.
Table 380.   bps Register
Bits Name Description
31:0 Bits per Color Sample The bits per color sample parameter. Returns a value of 10.
Table 381.   num_enginesRegister
Bits Name Description
31:0 Number of Engines The number of engines parameter. Returns a value of 1 or 2.
Table 382.   max_input_width Register
Bits Name Description
31:0 Maximum input video width The maximum input video width parameter. Returns a value of 2048 or 3840.
Table 383.   max_output_width Register
Bits Name Description
31:0 Maximum output video width The maximum output video width parameter. Returns a value of 2048 or 3840.
Table 384.   memory_buffer_size Register
Bits Name Description
31:0 Memory frame buffer size

The memory frame buffer size parameter. Returns a value of 0, 1 or 2.

0 = SD, 1=HD and 2=UHD
Table 385.   easy_warp Register
Bits Name Description
31:0 Use easy warp The Use easy warp parameter. Returns a value of 0 or 1.
Table 386.   int_control Register
Bits Name Description
0 Interrupt Enable Setting this bit to 1 will enable the interrupt. Setting to 0 will disable the interrupt.
Table 387.   int_status Register
Bits Name Description
0 Interrupt Status

Reading from this bit returns the status of the interrupt.

Writing a 1 to this bit will clear the interrupt. Once triggered, the interrupt will remain set until it is cleared by writing a 1 to this bit.

Debug and Measurement Registers

When you turn on Enable Debug Registers in the configuration settings of the Warp IP, a set of read only registers are available that provide various debug and measurement readings. The registers provide information useful during system bring up and debug and are accessed using the software API.

The available measurements are:

  • Input section:
    • INPUT_FRAME_COUNT
    • INPUT_FRAME_PERIOD
  • Output section
    • OUTPUT_FRAME_COUNT
    • OUTPUT_FRAME_PERIOD
  • Engine section
    • ENGINE_FRAME_COUNT
    • ENGINE_FRAME_PERIOD
    • ENGINE_BUSY_CYCLES
    • ENGINE_FRAME_DELAY
    • ENGINE_FAILED_TO_COMPLETE_FRAME
  • General section
    • MEM_WR_QUEUES0
    • MEM_WR_QUEUES1
    • MEM_RD_QUEUES0
    • MEM_RD_QUEUES1
Table 388.  Registers
Name Description
Input Section

The IP reads the debug registers relating to the input section of the Warp IP using the intel_vvp_warp_get_input_debug_register () API call.

INPUT_FRAME_COUNT

A count of the number of frames that have started at the input of the Warp IP. The last frame counted may be incomplete.

This counter can indicate the video activity at the input of the Warp IP. Sampling this count periodically indicates frame rate. A static count indicates that the upstream pipeline to the Warp IP is not producing video data.

INPUT_FRAME_PERIOD

The number of axi4s_vid_in_0_clock cycles seen between the last two start of frames seen at the input.

Use this counter to indicate the frame rate by comparing its value with the input clock frequency. If the value from this register varies greatly, it indicates an unstable processing pipeline upstream of the Warp IP.

Output Section

The debug registers relating to the output section of the Warp IP are read using the intel_vvp_warp_get_output_debug_register () API call.

OUTPUT_FRAME_COUNT

A count of the number of frames that have started at the output. The last frame counted may be incomplete.

This counter can indicate the video activity at the output of the Warp IP. Sampling this count periodically gives an indication of frame rate and a static count indicates that the downstream pipeline from the Warp IP is blocked and not accepting video data.

OUTPUT_FRAME_PERIOD

The number of axi4s_vid_out_0_clock cycles seen between the last two start of frames seen at the output.

This counter can indicate the frame rate by comparing its value with the output clock frequency. If the value from this register varies wildly, it indicates an unstable processing pipeline downstream of the Warp IP.

Engine Section

The debug registers relating to the engine section of the Warp IP are read using the intel_vvp_warp_get_engine_debug_register () API call.

ENGINE_FRAME_COUNT

A count of the number of frames that have started to be processed by the engine being referenced. The processing of the last frame counted may be in progress.

This counter is useful for indicating the video activity within the Warp IP. It will give indication that the Warp IP has been set up and that the engine has been configured to process correctly.

ENGINE_FRAME_PERIOD

The number of core_clock cycles seen between the last two start of frames seen at the engine.

This counter is useful for confirming that the core_clock is set correctly given the expected frame rate. It is also useful for comparing against the ENGINE_BUSY _CYCLES value as an indication of how busy the engine is.

ENGINE_BUSY CYCLES

The number of core_clock cycles for which the engine actively processes video data during the last frame. The counter starts when the processing of a frame begins and continues until either the IP completes the frame or encounters the start of the next frame. For the latter the engine fails to complete the processing of the frame in time, refer to the ENGINE_FAILED_TO_COMPLETE_FRAME status indicator.

When compared to the ENGINE_FRAME_PERIOD, the value of ENGINE_BUSY_CYCLES gives an indication of how busy an engine is during the processing of a frame.

ENGINE_FRAME_DELAY

The number of core_clock cycles between the start of the last input frame and the start of the engine’s processing phase.

Use this counter for low latency behavior to determine if you have enough delay when programming the low latency settings. It also indicates the input and output frame synchronization that is a requirement for correct low latency behavior. This delay must be stable for low latency IPs to work correctly. If the IP does not see a stable value, the delay is changing over time, which indicates the input to output frame synchronization may not be operating correctly.

ENGINE_FAILED_TO_COMPLETE_FRAME

A sticky flag that indicates that the engine has failed to complete the processing of a frame in the required frame period. The flag is cleared each time it is interrogated.

The IP sets this to indicate that the engine has not had sufficient time to process the video frame as required and image corruption occurs. The most likely reason is insufficient memory bandwidth available to the engine. Insufficient memory bandwidth gives processing stalls either while waiting for read data to return or stalls while blocked waiting for write data is sent out.

General Section

The debug registers relating to the general section of the Warp IP are read using the intel_vvp_warp_get debug_register () API call.

MEM_WR_QUEUES0

Indicates the number of internal requests that queue at any one time for each of the write accesses through the Warp IP’s memory controller. The IP stores each queue depth value as a byte in the 32-bit data word returned by the read to this register. The IP has a number of queues depending on the configuration (Table 389).

MEM_RD_QUEUES0 and MEM_RD_QUEUES1

Indicates the number of internal requests that queue at any one time for each of the read accesses through the Warp IP’s memory controller. The IP has two 32 bit registers for access to the read queue depths. Each individual queue depth value is stored as a nibble in the 32 bit data word returned by a read to one of these registers. The IP has a number of queues depending on the configuration (Table 390)

Table 389.  Separate Queue Values (Write Accesses)

The table shows the separate queue values for each configuration. The maximum depth value for each write queue is 48. If periodically sampling of the queue depth values finds them to be close to or at this maximum value, it indicates a limitation on write bandwidth from the IP to the external memory because the queues are filling up and blocking throughput.

Queue Easy Warp Single Engine Dual Engine
Video Input MEM_WR_QUEUES0[7:0] MEM_WR_QUEUES0[7:0] MEM_WR_QUEUES0[7:0]
Engine 0 Write N/A MEM_WR_QUEUES0[15:8] MEM_WR_QUEUES0[15:8]
Engine 1 Write N/A N/A MEM_WR_QUEUES0[23:16]
Table 390.  Separate Queue Values (Read Accesses)The table shows the separate queue values for each configuration.

The maximum value for each read queue depth is 12. If periodically sampling of queue depth values finds them close to or at this maximum value, it indicates a limitation on read bandwidth from the IP to the external memory because the queues are filling up and blocking throughput.

Queue Easy Warp Single Engine Dual Engine
Video Output MEM_RD_QUEUES0[3:0] MEM_RD_QUEUES0[3:0] MEM_RD_QUEUES0[3:0]
Engine 0 Video N/A MEM_RD_QUEUES0[7:4] MEM_RD_QUEUES0[7:4]
Engine 0 Coeff0 N/A MEM_RD_QUEUES0[11:8] MEM_RD_QUEUES0[11:8]
Engine 0 Coeff1 N/A MEM_RD_QUEUES0[15:12] MEM_RD_QUEUES0[15:12]
Engine 0 Coeff2 N/A MEM_RD_QUEUES0[19:16] MEM_RD_QUEUES0[19:16]
Engine 1 Video N/A N/A MEM_RD_QUEUES0[23:20]
Engine 1 Coeff0 N/A N/A MEM_RD_QUEUES0[27:24]
Engine 1 Coeff1 N/A N/A MEM_RD_QUEUES0[31:28]
Engine 1 Coeff2 N/A N/A MEM_RD_QUEUES1[3:0]
Level Two Title