Video and Vision Processing Suite Intel® FPGA IP User Guide

ID 683329
Date 9/30/2024
Public
Document Table of Contents
1. About the Video and Vision Processing Suite 2. Getting Started with the Video and Vision Processing IPs 3. Video and Vision Processing IPs Functional Description 4. Video and Vision Processing IP Interfaces 5. Video and Vision Processing IP Registers 6. Video and Vision Processing IPs Software Programming Model 7. Protocol Converter Intel® FPGA IP 8. 1D LUT Intel® FPGA IP 9. 3D LUT Intel® FPGA IP 10. Adaptive Noise Reduction Intel® FPGA IP 11. Advanced Test Pattern Generator Intel® FPGA IP 12. AXI-Stream Broadcaster Intel® FPGA IP 13. Bits per Color Sample Adapter Intel® FPGA IP 14. Black Level Correction Intel® FPGA IP 15. Black Level Statistics Intel® FPGA IP 16. Chroma Key Intel® FPGA IP 17. Chroma Resampler Intel® FPGA IP 18. Clipper Intel® FPGA IP 19. Clocked Video Input Intel® FPGA IP 20. Clocked Video to Full-Raster Converter Intel® FPGA IP 21. Clocked Video Output Intel® FPGA IP 22. Color Plane Manager Intel® FPGA IP 23. Color Space Converter Intel® FPGA IP 24. Defective Pixel Correction Intel® FPGA IP 25. Deinterlacer Intel® FPGA IP 26. Demosaic Intel® FPGA IP 27. FIR Filter Intel® FPGA IP 28. Frame Cleaner Intel® FPGA IP 29. Full-Raster to Clocked Video Converter Intel® FPGA IP 30. Full-Raster to Streaming Converter Intel® FPGA IP 31. Genlock Controller Intel® FPGA IP 32. Generic Crosspoint Intel® FPGA IP 33. Genlock Signal Router Intel® FPGA IP 34. Guard Bands Intel® FPGA IP 35. Histogram Statistics Intel® FPGA IP 36. Interlacer Intel® FPGA IP 37. Mixer Intel® FPGA IP 38. Pixels in Parallel Converter Intel® FPGA IP 39. Scaler Intel® FPGA IP 40. Stream Cleaner Intel® FPGA IP 41. Switch Intel® FPGA IP 42. Text Box Intel® FPGA IP 43. Tone Mapping Operator Intel® FPGA IP 44. Test Pattern Generator Intel® FPGA IP 45. Unsharp Mask Intel® FPGA IP 46. Video and Vision Monitor Intel FPGA IP 47. Video Frame Buffer Intel® FPGA IP 48. Video Frame Reader Intel FPGA IP 49. Video Frame Writer Intel FPGA IP 50. Video Streaming FIFO Intel® FPGA IP 51. Video Timing Generator Intel® FPGA IP 52. Vignette Correction Intel® FPGA IP 53. Warp Intel® FPGA IP 54. White Balance Correction Intel® FPGA IP 55. White Balance Statistics Intel® FPGA IP 56. Design Security 57. Document Revision History for Video and Vision Processing Suite User Guide

53.1.2. Warp IP Performance and Resource Utilization

Intel provides resource and utilization data for guidance. The designs target an Intel Arria 10 10AX115N2F40I2LG device or an Agilex™ 7 AGIB027R29A1E2V.

For devices other than Agilex™ 7 devices, the Warp IP supports clock rates of 300 MHz for the video related clocks axi4s_vid_in_0_clock, axi4s_vid_out_0_clock, and core_clock.

For Agilex™ 7 devices, the Warp IP supports clock rates of 600 MHz for the video related clocks axi4s_vid_in_0_clock, axi4s_vid_out_0_clock, and core_clock. This allows a single pixel in parallel, single engine configuration to process UHD frames at 60 fps. The Warp IP also supports a configuration of 2 pixels in parallel with one engine. Your design can process UHD frames at 60 fps on Agilex™ 7 devices with a reduced video clock rate of 300 MHz on the video input and output connections and running the main processing clock at 600 MHz.

Table 1089.   HD frame processing on Intel Arria 10 Device with Double Memory BounceProcessing frames of up to 1920x1080 resolution. Intel set the video related clocks axi4s_vid_in_0_clock, axi4s_vid_out_0_clock, and core_clock to a minimum of 150 MHz to allow the IP to process 60 fps. Set these clocks to 300 MHz for frame rates of 120 fps. Use mipmaps is off.
Pixel in Parallel Use Single Memory Bounce Number of Engines Max Video Width 176 177 Memory Buffer Size ALMs Memory Blocks (M20K) DSP Blocks
1 Off 1 2048 HD ~6,000 191 36
Table 1090.  HD frame processing on Intel Arria 10 Device with Single Memory BounceProcessing frames of up to 1920x1080 resolution. Intel set the video related clocksaxi4s_vid_in_0_clock, axi4s_vid_out_0_clock,and core_clockto a minimum of 150 MHz to allow the IP to process 60 fps. Set these clocks to 300 MHz for frame rates of 120 fps. Use mipmaps is off.
Pixel in Parallel Use Single Memory Bounce Cache Blocks per Engine Number of Engines Maximum Video Width 177 Memory BufferSize ALMs Memory Blocks (M20K) DSP Blocks
1 On 256 1 2048 HD ~6,000 163 36
1 On 512 1 2048 HD ~6,000 211 36
1 On 1024 1 2048 HD ~6,000 307 36
Table 1091.   UHD Frames at 30 fps on Intel Arria 10 Device with Double Memory Bounce Processing frames of up to 3840x2160 resolution at 30 fps. Intel set the video related clocks axi4s_vid_in_0_clock, axi4s_vid_out_0_clock, and core_clock to 300 MHz. Use mipmaps is off.
Pixel in parallel Use Single Memory Bounce Number of Engines Max Video Width 177 Memory Buffer Size ALMs Memory Blocks (M20K) DSP Blocks
1 Off 1 4096 UHD ~6,000 273 36
Table 1092.  UHD Frames at 60 fps on Intel Arria 10 Device with Double Memory BounceProcessing frames of up to 3840x2160 resolution at 60 fps. Intel set the video related clocks axi4s_vid_in_0_clock, axi4s_vid_out_0_clock, and core_clock to 300 MHz. Use mipmaps is off.
Pixel in parallel Use Single Memory Bounce Number of Engines Max Video Width177 Memory Buffer Size ALMs Memory Blocks (M20K) DSP Blocks
2 Off 2 4096 UHD ~10,000 354 72
Table 1093.  UHD frames at 60 fps on Intel Arria 10 Device with Single Memory BounceProcessing frames of up to 3840x2160 resolution at 60 fps. Intel set the video related clocks axi4s_vid_in_0_clock,axi4s_vid_out_0_clock,and core_clock to 300 MHz. Use mipmaps is off.
Pixel in parallel Use Single Memory Bounce Cache Blocks per Engine Number of Engines Max Video Width 177 Memory BufferSize ALMs Memory Blocks (M20K) DSP Blocks
2 On 256 2 4096 UHD ~10,000 300 72
2 On 512 2 4096 UHD ~10,000 396 72
2 On 1024 2 4096 UHD ~10,000 588 72
Table 1094.   One Pixel In Parallel UHD Frames at 60 fps, on Agilex™ 7 Device with Double Memory Bounce

Processing frames of up to 3840x2160 resolution at 60 fps. Intel set the video related clocks axi4s_vid_in_0_clock,axi4s_vid_out_0_clock,and core_clock to 600 MHz. .Use mipmaps is off.

Pixel in parallel Use Single Memory Bounce Number of Engines Max Video Width 177 Memory Buffer Size ALMs Memory Blocks (M20K) DSP Blocks
1 Off 1 4096 UHD ~8,000 249 36

Table 1095.  One Pixel In Parallel UHD Frames at 60 fps on Agilex™ 7 Device with Single Memory BounceProcessing frames of up to 3840x2160 resolution at 60 fps. Intel set the video related clocks axi4s_vid_in_0_clock,axi4s_vid_out_0_clock,andcore_clock to 600 MHz. Use mipmaps is off.
Pixel in parallel Use Single Memory Bounce Cache Blocks per Engine Number of Engines Max Video Width 177 Memory Buffer Size ALMs Memory Blocks (M20K) DSP Blocks
1 On 256 1 4096 UHD ~7,000 212 36
1 On 512 1 4096 UHD ~7,000 260 36
1 On 1024 1 4096 UHD ~7,000 356 36
Table 1096.   One Pixel In Parallel HD frame processing with Use easy warp on Intel Arria 10 Device

Processing frames of up to 1920x1080 resolution. Intel set the video related clocks axi4s_vid_in_0_clock and axi4s_vid_out_0_clock to a minimum of 150 MHz to allow the IP to process 60 fps. Set these clocks to 300 MHz for frame rates of 120 fps.

Pixel in parallel Maximum Video Width 177 Memory Buffer Size Use Easy Warp ALMs Memory Blocks (M20K) DSP Blocks
1 2048 HD On ~3,000 148 0
Table 1097.  One Pixel In Parallel UHD frame processing with Use easy warp on Intel Arria 10 Device  Processing frames of up to 3840 × 2160 resolution. Intel set the video related clocks axi4s_vid_in_0_clock, axi4s_vid_out_0_clock to a minimum of 300 MHz to allow the IP to process 30 fps.
Pixel in parallel Maximum Video Width 177 Memory Buffer Size Use Easy Warp ALMs Memory Blocks (M20K) DSP Blocks
1 3840 UHD On

~3000

271 0
Table 1098.  Two Pixels In Parallel UHD frame processing with Use easy warp on Intel Arria 10 Device  Processing frames of up to 3840 × 2160 resolution. Intel set the video related clocks axi4s_vid_in_0_clock, axi4s_vid_out_0_clock to a minimum of 300 MHz to allow the IP to process 60 fps. 
Pixel in parallel Easy Warp Maximum Video Width 177 Use Easy Warp Memory Buffer Size ALMs Memory Blocks (M20K) DSP Blocks
2 1 3840 On UHD

~3000

271 0
Table 1099.  UHD Frames at 120 fps, on Agilex 7 Device with Double Memory BounceProcessing frames of up to 3840x2160 resolution. Altera set the video related clocks axi4s_vid_in_0_clock,axi4s_vid_out_0_clock,and core_clockto 600 MHz with the memory clock av_mm_memory_host_clock set to 333 MHz. Use mipmaps is off. Four engines and four pixels in parallel with the 8K buffer size allow the Warp IP to process the bandwidth necessary for 120 fps.
Pixel in parallel Use Single Memory Bounce Number of Engines Max Video Width Memory Buffer Size ALMs Memory Blocks (M20K) DSP Blocks
4 Off 4 4096 8K ~23,000 629 144
Table 1100.  UHD Frames at 120 fps on Agilex 7 Device with Single Memory BounceProcessing frames of up to 3840x2160 resolution. Altera set the video related clocks axi4s_vid_in_0_clock, axi4s_vid_out_0_clock and core_clockto 600 MHz. Use mipmaps is off. Four engines and four pixels in parallel with the 8K buffer size allow the Warp IP to process the bandwidth necessary for 120 fps.
Pixel in parallel Use Single Memory Bounce Cache Blocks per Engine Number of Engines Max Video Width Memory Buffer Size ALMs Memory Blocks (M20K)

DSP

Blocks

4 On 256 4 4096 8K ~20,000 471 144
4 On 512 4 4096 8K ~21,000 663 144
4 On 1024 4 4096 8K ~22,000 1047 144
Table 1101.  8K Frames at 30 fps, on Agilex 7 Device with Double Memory BounceProcessing frames of up to 7680x4320 resolution. Altera set the video related clocks axi4s_vid_in_0_clock,axi4s_vid_out_0_clock,and core_clockto 600 MHz with the memory clock av_mm_memory_host_clock set to 333 MHz. Use mipmaps is off. Four engines and four pixels in parallel allow the Warp IP to process the bandwidth necessary for 8k at 30 fps.
Pixel in parallel Use Single Memory Bounce Number of Engines Max Video Width Memory Buffer Size ALMs Memory Blocks (M20K) DSP Blocks
4 Off 4 7680 8K ~24,000 737 144
Table 1102.  8K Frames at 30 fps on Agilex 7 Device with Single Memory BounceProcessing frames of up to 7680x4320 resolution. Altera set the video related clocks axi4s_vid_in_0_clock, axi4s_vid_out_0_clock and core_clockto 600 MHz. Use mipmaps is off. Four engines and four pixels in parallel allow the Warp IP to process the bandwidth necessary for 8K at 30 fps.
Pixel in parallel Use Single Memory Bounce Cache Blocksper Engine Number of Engines Max Video Width Memory Buffer Size ALMs Memory Blocks (M20K)

DSP

Blocks

4 On 256 4 7680 8K ~21,000 579 144
4 On 512 4 7680 8K ~22,000 771 144
4 On 1024 4 7680 8K ~23,000 1155 144
Table 1103.  Extra resources for Agilex 7 Devices with Mipmaps OnTurning Use mipmaps on requires extra resources that depend on the number of pixels in parallel, the maximum video width and the size of the memory buffers.
Pixel in parallel Max Video Width Memory Buffer Size ALMs Memory Blocks (M20K) DSP Blocks
1 4096 UHD ~2,000 131 18
4 4096 8K ~3,000 143 30
4 7680 8K ~4,000 259 30
Table 1104.  Extra resources for Arria 10 Devices with Mipmaps OnTurning Use mipmaps on requires extra resources that depend on the number of pixels in parallel, the maximum video width and the size of the memory buffers.
Pixel in parallel Max Video Width Memory Buffer Size ALMs Memory Blocks (M20K) DSP Blocks
1 2048 HD ~2,000 85 18
2 4096 UHD ~2,000 155 18
176 Same maximum video width for input and output.
177 Same maximum video width for input and output.