Video and Vision Processing Suite Intel® FPGA IP User Guide

ID 683329
Date 9/30/2024
Public
Document Table of Contents
1. About the Video and Vision Processing Suite 2. Getting Started with the Video and Vision Processing IPs 3. Video and Vision Processing IPs Functional Description 4. Video and Vision Processing IP Interfaces 5. Video and Vision Processing IP Registers 6. Video and Vision Processing IPs Software Programming Model 7. Protocol Converter Intel® FPGA IP 8. 1D LUT Intel® FPGA IP 9. 3D LUT Intel® FPGA IP 10. Adaptive Noise Reduction Intel® FPGA IP 11. Advanced Test Pattern Generator Intel® FPGA IP 12. AXI-Stream Broadcaster Intel® FPGA IP 13. Bits per Color Sample Adapter Intel® FPGA IP 14. Black Level Correction Intel® FPGA IP 15. Black Level Statistics Intel® FPGA IP 16. Chroma Key Intel® FPGA IP 17. Chroma Resampler Intel® FPGA IP 18. Clipper Intel® FPGA IP 19. Clocked Video Input Intel® FPGA IP 20. Clocked Video to Full-Raster Converter Intel® FPGA IP 21. Clocked Video Output Intel® FPGA IP 22. Color Plane Manager Intel® FPGA IP 23. Color Space Converter Intel® FPGA IP 24. Defective Pixel Correction Intel® FPGA IP 25. Deinterlacer Intel® FPGA IP 26. Demosaic Intel® FPGA IP 27. FIR Filter Intel® FPGA IP 28. Frame Cleaner Intel® FPGA IP 29. Full-Raster to Clocked Video Converter Intel® FPGA IP 30. Full-Raster to Streaming Converter Intel® FPGA IP 31. Genlock Controller Intel® FPGA IP 32. Generic Crosspoint Intel® FPGA IP 33. Genlock Signal Router Intel® FPGA IP 34. Guard Bands Intel® FPGA IP 35. Histogram Statistics Intel® FPGA IP 36. Interlacer Intel® FPGA IP 37. Mixer Intel® FPGA IP 38. Pixels in Parallel Converter Intel® FPGA IP 39. Scaler Intel® FPGA IP 40. Stream Cleaner Intel® FPGA IP 41. Switch Intel® FPGA IP 42. Text Box Intel® FPGA IP 43. Tone Mapping Operator Intel® FPGA IP 44. Test Pattern Generator Intel® FPGA IP 45. Unsharp Mask Intel® FPGA IP 46. Video and Vision Monitor Intel FPGA IP 47. Video Frame Buffer Intel® FPGA IP 48. Video Frame Reader Intel FPGA IP 49. Video Frame Writer Intel FPGA IP 50. Video Streaming FIFO Intel® FPGA IP 51. Video Timing Generator Intel® FPGA IP 52. Vignette Correction Intel® FPGA IP 53. Warp Intel® FPGA IP 54. White Balance Correction Intel® FPGA IP 55. White Balance Statistics Intel® FPGA IP 56. Design Security 57. Document Revision History for Video and Vision Processing Suite User Guide

17.3. Chroma Resampler IP Functional Description

Chroma resampler algorithms

The chroma resampler implements four basic resampling operations:

  • Horizontal upsampling: 4:2:2 -> 4:4:4
  • Horizontal downsampling: 4:4:4 -> 4:2:2
  • Vertical upsampling: 4:2:0 -> 4:2:2
  • Vertical downsampling: 4:2:2 -> 4:2:0

For conversions between 4:4:4 and 4:2:0 the relevant horizontal and vertical sampling operations chain automatically to achieve the correct end result. The chroma resampler supports three different algorithms for each resampling operation; nearest neighbor, bilinear and filtered. The three algorithms vary in the level of visual quality provided, the chroma siting, and the resources for implementation.

Nearest Neighbor

Nearest neighbor is the lowest quality resampling algorithm, with the lowest device resource usage. For horizontal downsampling (4:4:4 -> 4:2:2) it drops every other Cb and Cr sample within the video line. For horizontal upsampling (4:2:2 -> 4:4:4) it repeats each Cb and Cr sample. The nearest neighbor algorithm uses left siting (co-siting) for the 4:2:2 chroma samples – both the Cb and Cr samples from the even indexed Y samples are retained during downsampling.

Figure 40. Nearest Neighbor

The nearest neighbor algorithm works in the same way for vertical resampling. For vertical downsampling (4:2:2 -> 4:2:0) the IP discards the chroma data from every other video line. For upsampling (4:2:0 -> 4:2:2) the IP repeats the chroma data for two lines of luma data. The nearest neighbor algorithm uses top siting (cositing) for both the Cb and Cr planes. The IP preserves chroma data from lines 0, 2, 4, etc in downsampling. The IP discards the data from lines 1, 3, 5, etc.

Figure 41. Nearest Neighbor
Bilinear

The bilinear algorithm offers a midpoint between visual image quality and device resource cost. You can parameterize the bilinear algorithm to use either left or center horizontal siting of the chroma data, and top or centre vertical siting of the chroma data.

Top or left chroma siting is the simpler of the two algorithm options. Top or left downsampling using the bilinear algorithm is exactly equivalent to nearest neighbor downsampling. The IP discards the chroma data from the odd indexed pixels. Top or left upsampling recreates the chroma data for odd indexed pixels by taking the average of the chroma samples for the two neighboring even indexed pixels. The IP takes the chroma values for the even indexed pixels directly from the input values. Both the nearest neighbor algorithm and the bilinear algorithm with top or left chroma siting are fully reversible. The upsampling algorithm creates new data without altering the values of the any the original input data. The downsampling algorithm simply discards the values that the upsampling algorithm creates.

Centre chroma sampling for the bilinear algorithm is more complex and is not reversible because the IP alters all of the chroma values during both the upsample and downsample processes.
Figure 42. BilinearThe figure and equations show how the IP calculates the bilinear resampled chroma for horizontal upsampling (4:2:4 -> 4:4:4) and downsampling (4:4:4 -> 4:2:2) using center chroma siting.

The bilinear algorithm using center siting works in the same way for vertical upsampling (4:2:0 -> 4:2:2) and downsampling (4:2:2 -> 4:2:0).

Filtered

The filtered algorithm is the most computationally expensive and device resource intense algorithm, but it offers increased visual quality. You can parameterize the filtered algorithm to use either left or center horizontal siting of the chroma data, and top or center vertical siting of the chroma data.

For downsampling conversions (4:2:2 -> 4:2:0 and 4:4:4 -> 4:2:2) the filtered algorithm applies an 8-tap Lanczos2 resampling filter to generate the downsampled data. The IP applies different phase shifts to the Lanczos2 function when generating the coefficients for each chroma siting option. For top and left chroma siting, the IP applies a phase shift of 0 pixels to the Lanczos2 coefficients. For center chroma siting the phase shift is +0.5 pixels.

For upsampling conversions (4:2:0 -> 4:2:2 and 4:2:2 -> 4:4:4) the filtered algorithm applies a 4-tap Lanczos2 resampling filter to generate the upsampled data. For top/left chroma siting, phase shifts of 0 and 0.5 pixels are applied to the Lanczos2 coefficients for the even and odd indexed chroma samples respectively.For center chroma siting the phases shifts are -0.25 and +0.25 pixels.

You can also turn on luma adaption for upsampling conversions. This feature further increases device resource usage but may reduce color bleed around edges when compared to the default filtered algorithm. The feature is the only chroma resampler to implement some logic in DSP blocks. When you turn on luma adaption, the IP compares differences between successive luma samples to an edge threshold to detect significant edges. In areas where the IP detects edges, you can shift the phase of the Lanczos-2 filter by up to 0.25 pixels to the left or right. This shift weights the resulting chroma samples more intensely towards the more appropriate side of the edge.

Color planes per pixel

Most Video and Vision Processing IPs allow you to select the number of color planes per pixel, but the chroma resampler does not have that parameter and sets the value automatically. The number of color planes per pixel is a function of the chroma resampling conversions that you select, which determine the chroma sampling formats that are supported at the input and output interfaces. 4:4:4 and 4:2:0 chroma samplings both require three color panes per pixel. 4:2:2 only requires 2 color planes. 4:2:2 can be transmitted through a link with 3 color planes, with the planes in the most significant bits of each pixel unused.

If you select any of the following chroma sampling conversions, the input interface uses three color planes per pixel, otherwise it uses two color planes:

  • 4:2:0 passthrough
  • 4:2:0 to 4:4:4
  • 4:2:0 to 4:2:2
  • 4:4:4 passthrough
  • 4:4:4 to 4:2:2
  • 4:4:4 to 4:2:0

If you select any of the following chroma sampling conversions, the output interface uses three color planes per pixel, otherwise it uses two color planes:

  • 4:2:0 passthrough
  • 4:2:0 to 4:4:4
  • 4:4:4 passthrough
  • 4:4:4 to 4:2:0
  • 4:2:2 to 4:4:4
  • 4:2:2 to 4:2:0