1.5. Stall Behavior and Error Recovery

Video and Image Processing Suite User Guide

Download PDF

ID 683416

Date 4/04/2022

Version

Public

Visible to Intel only — GUID: bhc1411019841314

Ixiasoft

View Details

1.5. Stall Behavior and Error Recovery

The Video and Image Processing Suite IPs do not continuously process data. Instead, they use flow-controlled Avalon streaming interfaces, which allow them to stall the data while they perform internal calculations.

During control packet processing, the IPs might stall frequently and read or write less than once per clock cycle. During data processing, the IPs generally process one input or output per clock cycle. The IPs have some stalling cycles. Typically, the stalling cycles are for internal calculations between rows of image data and between frames/fields.

When stalled, an IP indicates that it is not ready to receive or produce data. The time spent in the stalled state varies between IPs and their parameterizations. In general, it is a few cycles between rows and a few more between frames.

If data is not available at the input when required, all of the IPs stall and do not output data. With the exceptions of the Deinterlacer and Frame Buffer in double or triple-buffering mode, none of the IPs overlap the processing of consecutive frames. The first sample of frame F + 1 is not input until after the IPs produce the last sample of frame F.

When the IPs receive an endofpacket signal unexpectedly (early or late), the IPs recover from the error and prepare for the next valid packet (control or data).

IP Core	Stall Behavior	Error Recovery
2D FIR Filter II	Has a delay of a little more than N–1 lines between data input and output in the case of a N×N 2D FIR Filter. Delay caused by line buffering internal to the IP.	An error condition occurs if an `endofpacket` signal is received too early or too late for the run-time frame size. In either case, the 2D FIR Filter always creates output video packets of the configured size. If an input video packet has a late `endofpacket` signal, then the extra data is discarded. If an input video packet has an early `endofpacket` signal, then the video frame is padded with an undefined combination of the last input pixels.
Mixer II	All modes stall for a few cycles after each output frame and between output lines. Between frames, the IP processes non-image data packets from its input layers in sequential order. The IP may exert backpressure during the process until the image data header has been received for all its input. During the mixing of a frame, the IP: Reads from the background input for each non-stalled cycle. Reads from the input ports associated with layers that currently cover the background image. Because of pipelining, the foreground pixel of layer N is read approximately N active cycles after the corresponding background pixel has been read. If the output is applying backpressure or if one input is stalling, the pipeline stalls and the backpressure propagates to all active inputs. When alpha blending is enabled, one data sample is read from each alpha port once each time that a whole pixel of data is read from the corresponding input port. There is no internal buffering in the IP, so the delay from input to output is just a few clock cycles and increases linearly with the number of inputs.	The Mixer II IP core processes video packets from the background layer until the end of packet is received. Receiving an `endofpacket` signal too early for the background layer—the IP core enters error mode and continues writing data until it has reached the end of the current line. The `endofpacket` signal is then set with the last pixel sent. Receiving an `endofpacket` signal early for one of the foreground layers or for one of the alpha layers—the IP core stops pulling data out of the corresponding input and pads the incomplete frame with undefined samples. Receiving an `endofpacket` signal late for the background layer, one or more foreground layers, or one or more alpha layers—the IP core enters error mode. This error recovery process maintains the synchronization between all the inputs and is started once the output frame is completed. A large number of samples may have to be discarded during the operation and backpressure can be applied for a long time on most input layers. Consequently, this error recovery mechanism could trigger an overflow at the input of the system.
Avalon-ST Video Stream Cleaner	All modes stall for a few cycles between frames and between lines.	Receiving an early `endofpacket` signal—the IP core stalls its input but continues writing data until it has sent an entire frame. Not receiving an `endofpacket` signal at the end of a frame—the IP core discards data until it finds end-of-packet.
Chroma Resampler II	All modes stall for a few cycles between frames and between lines. Latency from input to output varies depending on the operation mode of the IP core. The only modes with latency of more than a few cycles are 4:2:0 to 4:2:2 and 4:2:0 to 4:4:4—corresponding to one line of 4:2:0 data The quantities of data input and output are not equal because this is a rate-changing function. Always produces the same number of lines that it accepts—but the number of samples in each line varies according to the subsampling pattern used. When not stalled, always processes one sample from the more fully sampled side on each clock cycle. For example, the subsampled side pauses for one third of the clock cycles in the 4:2:2 case or half of the clock cycles in the 4:2:0 case.	Receiving an early `endofpacket` signal—the IP core stalls its input but continues writing data until it has sent an entire frame. Not receiving an `endofpacket` signal at the end of a frame—the IP core discards data until it finds end-of-packet.
Clipper II	Stalls for a few cycles between lines and between frames. Internal latency is less than 10 cycles. During the processing of a line, it reads continuously but only writes when inside the active picture area as defined by the clipping window.	Receiving an early `endofpacket` signal—the IP core stalls its input but continues writing data until it has sent an entire frame. Not receiving an `endofpacket` signal at the end of a frame—the IP core discards data until it finds end of packet.
Clocked Video Input II ³	Dictated by incoming video. If its output FIFO is empty, during horizontal and vertical blanking periods the IP core does not produce any video data.	If an overflow is caused by a downstream core failing to receive data at the rate of the incoming video, the Clocked Video Input sends an `endofpacket` signal and restart sending video data at the start of the next frame or field.
Clocked Video Output II	Dictated by outgoing video. If its input FIFO is full, during horizontal and vertical blanking periods the IP stalls and does not take in any more video data.	Receiving an early `endofpacket` signal— the IP resynchronizes the outgoing video data to the incoming video data on the next start of packet it receives. Receiving a late `endofpacket`— the IP resynchronizes the outgoing video data to the incoming video immediately.
Color Plane Sequencer II	Stalls for a few cycles between frames and user/control packets The Avalon-ST Video transmission settings (color planes in sequence/parallel, number of color planes and number of pixels per beat) determine the throughput for each I/O. The slowest interface limits the overall rate of the others	Processes video packets until the IP core receives an `endofpacket` signal on either inputs. Frame dimensions taken from the control packets are not used to validate the sizes of the input frames.. When receiving an `endofpacket` signal on either `din0` or `din1`; the IP core terminates the current output frame. When both inputs are enabled and the `endofpacket` signals do not line up, extra input data on the second input is discarded until the end of packet is signaled.
Color Space Converter II	Only stalls between frames and not between rows. It has no internal buffering apart from the registers of its processing pipeline—only a few clock cycles of latency.	Processes video packets until the IP core receives an `endofpacket` signal—the control packets are not used. Any mismatch of the `endofpacket` signal and the frame size is propagated unchanged to the next IP core.
Control Synchronizer	Stalls for several cycles between packets. Stalls when it enters a triggered state while it writes to the Avalon-MM Slave ports of other IP cores. If the slaves do not provide a wait request signal, the stall lasts for no more than 50 clock cycles. Otherwise the stall is of unknown length.	Processes video packets until the IP core receives an `endofpacket` signal—the image width, height and interlaced fields of the control data packets are not compared against the following video data packet. Any mismatch of the `endofpacket` signal and the frame size of video data packet is propagated unchanged to the next IP core.
Deinterlacer II	Stores input video fields in the external memory and concurrently uses these input video fields to construct deinterlaced frames. Stalls up to 50 clock cycles for the first output frame. Additional delay of one line for second output frame because the IP core generates the last line of the output frame before accepting the first line of the next input field. Delay of two lines for the following output frames, which includes the one line delay from the second output frame. For all subsequent fields, the delay alternates between one and two lines.	Bob and Weave configurations always recover from an error caused by illegal control or video packets. Motion adaptive modes require the embedded stream cleaner to be enabled to fully recover from errors.
Frame Buffer II	May stall frequently and read or write less than once per clock cycle during control packet processing. During data processing at the input or at the output, the stall behavior of the IP core is largely decided by contention on the memory bus.	Does not rely on the content of the control packets to determine the size of the image data packets. Any early or late `endofpacket` signal and any mismatch between the size of the image data packet and the content of the control packet are propagated unchanged to the next IP core. Does not write outside the memory allocated for each non-image and image Avalon-ST video packet—packets are truncated if they are larger than the maximum size defined at compile time.
Gamma Corrector II	Stalls only between frames and not between rows. Has no internal buffering aside from the registers of its processing pipeline— only a few clock cycles of latency	Processes video packets until the IP core receives an `endofpacket` signal—non-image packets are propagated but the content of control packets is ignored. Any mismatch of the `endofpacket` signal and the frame size is propagated unchanged to the next IP core.
Interlacer II	Alternates between propagating and discarding a row from the input port while producing an interlaced output field—the output port is inactive every other row. The delay from input to output is a few clock cycles when pixels are propagated.	Receiving `endofpacket` signal later than expected—discards extra data. Receiving an early `endofpacket` signal—the current output field is interrupted as soon as possible and may be padded with a single undefined pixel.
Scaler II	The ratio of reads to writes is proportional to the scaling ratio and occurs on both a per-pixel and a per-line basis. The frequency of lines where reads and writes occur is proportional to the vertical scaling ratio. For example, scaling up vertically by a factor of 2 results in the input being stalled every other line for the length of time it takes to write one line of output; scaling down vertically by a factor of 2 results in the output being stalled every other line for the length of time it takes to read one line of input. In a line that has both input and output active, the ratio of reads and writes is proportional to the horizontal scaling ratio. For example, scaling from 64×64 to 128×128 causes 128 lines of output, where only 64 of these lines have any reads in them. For each of these 64 lines, there are two writes to every read. The internal latency of the IP core depends on the scaling algorithm and whether any run time control is enabled. The scaling algorithm impacts stalling as follows: Bilinear mode: a complete line of input is read into a buffer before any output is produced. At the end of a frame there are no reads as this buffer is drained. The exact number of possible writes during this time depends on the scaling ratio. Polyphase mode with N _v vertical taps: N _v – 1 lines of input are read into line buffers before any output is ready. The scaling ratio depends on the time at the end of a frame where no reads are required as the buffers are drained. Enabling run-time control of resolutions affects stalling between frames: With no run-time control: about 10 cycles of delay before the stall behavior begins, and about 20 cycles of further stalling between each output line. With run-time control of resolutions: about additional 25 cycles of delay between frames.	Receiving an early `endofpacket` signal at the end of an input line—the IP core stalls its input but continues writing data until it has sent one further output line. Receiving an early `endofpacket` signal part way through an input line—the IP core stalls its input for as long as it takes for the open input line to complete; completing any output line that may accompany that input line. Then continues to stall the input, and writes one further output line. Not receiving an `endofpacket` signal at the end of a frame—the IP core discards extra data until it finds an end of packet.
Switch II	Only stalls its inputs when performing an output switch. Before switching its outputs, the IP core synchronizes all its inputs and the inputs may be stalled during this synchronization.	—
Test Pattern Generator II	All modes stall for a few cycles after a field control packet, and between lines. When producing a line of image data, the IP core produces one sample output on every clock cycle, but it can be stalled without consequences if other functions down the data path are not ready and exert backpressure.	—
Warp Lite	All modes produce a few cycles of stall if the line store fills. The line store size ensures it shouldn’t fill unless a mismatch occurs between input and output rates.	The Warp Lite IP has a built in stream cleaner to handle error conditions. Receiving an `endofpacket` signal too early causes the stream cleaner to pad the packet up to the frame size specified by the preceding control packet. Receiving an `endofpacket` signal too late causes the stream cleaner to clip the packet to the frame size specified by the preceding control packet. Receiving a video packet without a preceding control packet causes the stream cleaner to discard the video packet.

³ For CVI II IP, the error recovery behavior varies depending on the Platform Designer parameters. Refer to the Clocked Video Interface IP for more information.

Select Your Language

Using Intel.com Search

Quick Links

Recent Searches

Advanced Search

Only search in

Video and Image Processing Suite User Guide

1.5. Stall Behavior and Error Recovery