Video and Vision Processing Suite Intel® FPGA IP User Guide

ID 683329
Date 9/30/2024
Public
Document Table of Contents
1. About the Video and Vision Processing Suite 2. Getting Started with the Video and Vision Processing IPs 3. Video and Vision Processing IPs Functional Description 4. Video and Vision Processing IP Interfaces 5. Video and Vision Processing IP Registers 6. Video and Vision Processing IPs Software Programming Model 7. Protocol Converter Intel® FPGA IP 8. 1D LUT Intel® FPGA IP 9. 3D LUT Intel® FPGA IP 10. Adaptive Noise Reduction Intel® FPGA IP 11. Advanced Test Pattern Generator Intel® FPGA IP 12. AXI-Stream Broadcaster Intel® FPGA IP 13. Bits per Color Sample Adapter Intel® FPGA IP 14. Black Level Correction Intel® FPGA IP 15. Black Level Statistics Intel® FPGA IP 16. Chroma Key Intel® FPGA IP 17. Chroma Resampler Intel® FPGA IP 18. Clipper Intel® FPGA IP 19. Clocked Video Input Intel® FPGA IP 20. Clocked Video to Full-Raster Converter Intel® FPGA IP 21. Clocked Video Output Intel® FPGA IP 22. Color Plane Manager Intel® FPGA IP 23. Color Space Converter Intel® FPGA IP 24. Defective Pixel Correction Intel® FPGA IP 25. Deinterlacer Intel® FPGA IP 26. Demosaic Intel® FPGA IP 27. FIR Filter Intel® FPGA IP 28. Frame Cleaner Intel® FPGA IP 29. Full-Raster to Clocked Video Converter Intel® FPGA IP 30. Full-Raster to Streaming Converter Intel® FPGA IP 31. Genlock Controller Intel® FPGA IP 32. Generic Crosspoint Intel® FPGA IP 33. Genlock Signal Router Intel® FPGA IP 34. Guard Bands Intel® FPGA IP 35. Histogram Statistics Intel® FPGA IP 36. Interlacer Intel® FPGA IP 37. Mixer Intel® FPGA IP 38. Pixels in Parallel Converter Intel® FPGA IP 39. Scaler Intel® FPGA IP 40. Stream Cleaner Intel® FPGA IP 41. Switch Intel® FPGA IP 42. Text Box Intel® FPGA IP 43. Tone Mapping Operator Intel® FPGA IP 44. Test Pattern Generator Intel® FPGA IP 45. Unsharp Mask Intel® FPGA IP 46. Video and Vision Monitor Intel FPGA IP 47. Video Frame Buffer Intel® FPGA IP 48. Video Frame Reader Intel FPGA IP 49. Video Frame Writer Intel FPGA IP 50. Video Streaming FIFO Intel® FPGA IP 51. Video Timing Generator Intel® FPGA IP 52. Vignette Correction Intel® FPGA IP 53. Warp Intel® FPGA IP 54. White Balance Correction Intel® FPGA IP 55. White Balance Statistics Intel® FPGA IP 56. Design Security 57. Document Revision History for Video and Vision Processing Suite User Guide

39.3.5. 422 and 420 Chroma Sampled Data Scaling

The Intel FPGA streaming video protocol that the scaler IP uses to transmit and receive video fields allows for the chroma sampling of the video to vary from frame to frame. For most of the IPs in the Video and Vision Processing suite, support for variable chroma sampling adds little or no extra resource cost. Variable chroma sampling of the video is always on and not gated by any parameters.

However, support for scaling 444, 422 and 420 chroma sampling on a field-variable basis adds significant ALM and memory cost to the resource footprint of the scaler. The scaler includes three parameters (444 chroma sampling, 422 chroma sampling and 420 chroma sampling) to allow you to select which chroma sampling formats the IP should support. Intel recommends that you only turn on chroma samplings formats that your scaler expects to receive in your system.

When the scaler processes 444 sampled data, each color plane is scaled independently, and each color plane has the same input and output dimensions. When the scaler processes 422 sampled data, each color plane is still processed independently (using the same filters as the 444 case), but the width of the chroma planes at the input and output is half that of the luma plane. Likewise, for 420 sampled data, each color plane is scaled independently, and the width and height of the chroma planes is half that of the luma plane. In all cases, the scaling ratio applied to each color plane must be the same. The scaler requires that the input and output field width are even for 422 sampled data, and the input and output width and height are even for 420 sampled data.

For 420 sampled data, each video line contains data from either the blue chroma (Cb) or red chroma (Cr) color plane. To supply both N lines of Cb and Cr data to an N tap vertical scaling filter, 2N lines of video data must be stored in line buffer, which is twice the storage required to process 444 or 422 sampled data. To reduce the on-chip memory footprint of the scaler with 420 chroma sampling enabled, you can turn on Mirror 420 chroma data. When on, the size of the line buffer is reduced back the N lines (for an N tap filter) and the IP mirrors N/2 lines of Cb and Cr data available in the buffer to fill the available taps. This parameter might give a small reduction in the quality of the scaled results for 420 chroma sampled data. You must assess any reduction in memory usage against the potential reduction in quality.

For 420 sampled data, the IP packs two luma samples into the three color planes that make up a standard pixel for either 444 or 422 chroma sampling. The scaler applies the horizontal scaling algorithm to the luma plane independently of the chroma planes. To allow for the double packing of the luma samples the scaler must, by default, implement the luma scaling with processing for twice the number of pixels in parallel you specify. However, you might not require the full processing bandwidth of the interface using 420 sampling. For example, if you have an HDMI 2.0 interface that supports up to 4k60 processing in both 444 and 420 sampling modes, the 420 mode only uses half the bandwidth allowed by the interface. In this example, the scaler need not use double the pixels in parallel to process the luma data and you should turn on the Half rate 420 parameter. This parameter specifies that the IP does not require internal doubling of pixels in parallel for processing the luma plane.