Academia.eduAcademia.edu

Figure 3 – uploaded by Peter Pirsch

See full PDFdownloadDownload figure
Fig. 2.. Motion estimation by block matching.

Figure 2 . Motion estimation by block matching.

Related Figures (40)

Table 2. Video Coding Standards communications with standardized low bit rate channels. International committees have been working for several years onto world wide image coding standards. According to the requirements of the applications different image formats and coding schemes have been specified. The major characteristics of image formats and coding standards are listed in Table 1 and Table 2. This tables are not complete. They should give just some indication of image formats, source rates, application fields and coded bit rates. A large range for the source rates of digitized video and coded bit rates can be recognized. There are coding schemes for still pictures (JPEG), noninterlaced video sequences (H.261, MPEG-1) and interlaced video sequences (MPEG-2). The coding schemes have a large variety of parameters in order to cover the requirements of a wide range of applications. In particular the MPEG-2 standard is a generic approach with different profiles and levels. In the following the basics of the JPEG and MPEG coding schemes will be explained.
Table 2. Video Coding Standards communications with standardized low bit rate channels. International committees have been working for several years onto world wide image coding standards. According to the requirements of the applications different image formats and coding schemes have been specified. The major characteristics of image formats and coding standards are listed in Table 1 and Table 2. This tables are not complete. They should give just some indication of image formats, source rates, application fields and coded bit rates. A large range for the source rates of digitized video and coded bit rates can be recognized. There are coding schemes for still pictures (JPEG), noninterlaced video sequences (H.261, MPEG-1) and interlaced video sequences (MPEG-2). The coding schemes have a large variety of parameters in order to cover the requirements of a wide range of applications. In particular the MPEG-2 standard is a generic approach with different profiles and levels. In the following the basics of the JPEG and MPEG coding schemes will be explained.
Table 1 Digital Video Formats VLSI. Recent advances in VLSI technology relieve the hardware problems but the high demands of video coding require special architectural approaches adapted to the video signal processing schemes. This paper provides characteris- tics of application specific integrated components for video coding and discusses the applied architectural structures.
Table 1 Digital Video Formats VLSI. Recent advances in VLSI technology relieve the hardware problems but the high demands of video coding require special architectural approaches adapted to the video signal processing schemes. This paper provides characteris- tics of application specific integrated components for video coding and discusses the applied architectural structures.
Fig. 1. Baseline JPEG coder-decoder.
Fig. 1. Baseline JPEG coder-decoder.
Fig. 3. MPEG encoder-decoder. input data. In general, for reduction of hardware complexity the reconstructed image from the frame memory in the feed- back loop is used for motion estimation. The block diagram in Fig. 3 specifies the basic structure of the H.261, MPEG- 1, and MPEG-2 encoder-decoder [1], [3], [4]. The H.261 codec has in addition a special loop filter for reduction of quantization effects on prediction.
Fig. 3. MPEG encoder-decoder. input data. In general, for reduction of hardware complexity the reconstructed image from the frame memory in the feed- back loop is used for motion estimation. The block diagram in Fig. 3 specifies the basic structure of the H.261, MPEG- 1, and MPEG-2 encoder-decoder [1], [3], [4]. The H.261 codec has in addition a special loop filter for reduction of quantization effects on prediction.
Fig. 4. MPEG temporal predictive coding.
Fig. 4. MPEG temporal predictive coding.
The discrete cosine transform (DCT) is a real-valued fre- quency transform similar to the discrete Fourier transform (DFT). The DCT has first been proposed by Ahmed et al. [9]. A recent book [85] provides an extensive introduction and an in-depth analysis of the properties, the various algorithms and the applications of the DCT. Our goal in this chapter is to focus on the implementation issues of the DCT. When applied to an image block of size Lx L, the two dimensional DCT (2D-DCT) can be expressed as follows:
The discrete cosine transform (DCT) is a real-valued fre- quency transform similar to the discrete Fourier transform (DFT). The DCT has first been proposed by Ahmed et al. [9]. A recent book [85] provides an extensive introduction and an in-depth analysis of the properties, the various algorithms and the applications of the DCT. Our goal in this chapter is to focus on the implementation issues of the DCT. When applied to an image block of size Lx L, the two dimensional DCT (2D-DCT) can be expressed as follows:
Fig. 6. Separated DCT implementation according to [23]. products, the silicon area optimization brought by dedi- cated architectures, compared to programmable architec- tures, leads to lower production cost.
Fig. 6. Separated DCT implementation according to [23]. products, the silicon area optimization brought by dedi- cated architectures, compared to programmable architec- tures, leads to lower production cost.
Fig. 7. B.G. Lee fast flowgraph for the 8 points DCT [14].
Fig. 7. B.G. Lee fast flowgraph for the 8 points DCT [14].
Computing separate horizontal and vertical 1D-DCT’s of length E directly from (10) requires 2L? multiplications and additions. Most people use this property for reducing the computational cost, compared to a direct 2D implementa- tion. Several realizations applying a direct implementation of the separable DCT have been reported [23], [13], [18]. by performing successively Z one-dimensional DCT’s (1D- DCT’s) on the rows and then Z 1D-DCT’s on the resulting columns. An £-point 1D-DCT can be expressed as follows:
Computing separate horizontal and vertical 1D-DCT’s of length E directly from (10) requires 2L? multiplications and additions. Most people use this property for reducing the computational cost, compared to a direct 2D implementa- tion. Several realizations applying a direct implementation of the separable DCT have been reported [23], [13], [18]. by performing successively Z one-dimensional DCT’s (1D- DCT’s) on the rows and then Z 1D-DCT’s on the resulting columns. An £-point 1D-DCT can be expressed as follows:
As a first step, it is well known that, for even L, an L-point DCT can be decomposed into two (LZ/2) x (L£/2) matrix- vector products. Thus the computation cost is reduced. As an example, an 8-point DCT can be computed in two separate parts: the 8-points DCT matrix can be computed as denoted in (12).
As a first step, it is well known that, for even L, an L-point DCT can be decomposed into two (LZ/2) x (L£/2) matrix- vector products. Thus the computation cost is reduced. As an example, an 8-point DCT can be computed in two separate parts: the 8-points DCT matrix can be computed as denoted in (12).
More generally, the matrices in (13) and (14) can be decomposed in a number of simpler matrices, the compo- sition of which can be expressed as a flowgraph. Many fast algorithms have been proposed. Fig. 7 illustrates the flowgraph of the B. G. Lee’s algorithm, which is commonly used [14]. Several implementation using fast flowgraphs have been reported [10], [12].
More generally, the matrices in (13) and (14) can be decomposed in a number of simpler matrices, the compo- sition of which can be expressed as a flowgraph. Many fast algorithms have been proposed. Fig. 7 illustrates the flowgraph of the B. G. Lee’s algorithm, which is commonly used [14]. Several implementation using fast flowgraphs have been reported [10], [12].
Fig. 8. Architecture of a Mf input inner product using distributed arithmetic.
Fig. 8. Architecture of a Mf input inner product using distributed arithmetic.
where {c;} are N-b constants and {x;} are coded in B bits in 2’s complement. Then (15) can be rewritten as The change of summing order in 7 and in j char- acterizes the distributed arithmetic scheme in which the initial multiplications are distributed to another computation pattern. Since the term C; has only 2” possible values (which depend on the 2;,; values), it is possible to store these 2” possible values in a ROM. An input set of M bits {z0,;,21,;,22,j,---,2a-1,j} is used as an address, allowing to retrieve the C; values. These intermediate results are accumulated in B clock cycles, for producing one Y value. Fig. 8 shows a typical architecture for the computation of a M input inner product. The inverter and the MUX are used for inverting the final output of the ROM in order to compute Cp.
where {c;} are N-b constants and {x;} are coded in B bits in 2’s complement. Then (15) can be rewritten as The change of summing order in 7 and in j char- acterizes the distributed arithmetic scheme in which the initial multiplications are distributed to another computation pattern. Since the term C; has only 2” possible values (which depend on the 2;,; values), it is possible to store these 2” possible values in a ROM. An input set of M bits {z0,;,21,;,22,j,---,2a-1,j} is used as an address, allowing to retrieve the C; values. These intermediate results are accumulated in B clock cycles, for producing one Y value. Fig. 8 shows a typical architecture for the computation of a M input inner product. The inverter and the MUX are used for inverting the final output of the ROM in order to compute Cp.
Table 3 Computation and Storage Requirements for Different DCT Algorithms (ZL + Z DCT) The through-put rate Ry in Mpel/s can be transferred to the computational rate Rc with Nop,pet. For an 8 x 8 separable 2D DCT based on matrix vector multiplication, this results in
Table 3 Computation and Storage Requirements for Different DCT Algorithms (ZL + Z DCT) The through-put rate Ry in Mpel/s can be transferred to the computational rate Rc with Nop,pet. For an 8 x 8 separable 2D DCT based on matrix vector multiplication, this results in
candidate block within the search window. The distortion D is computed for all the (2w + 1)’ possible positions of the candidate block within the search window in (21) and the block corresponding to the minimum distortion is used for prediction. The position of this block within the search window is represented by the motion vector v, see (21). a search window, but several other approaches have been reported in order to reduce the computation requirements by using an “intelligent” or “directed” search [26], [29], [30], [34], [36], [39], [40]. oe ae Ree ee eee eee oe 1) Exhaustive Search Block Matching: Applying an ex- haustive search block matching algorithm, a block of size Nx\N of the current image (reference block, denoted X) is matched with all the blocks located within a search window (candidate blocks, denoted Y). The maximum displacement will be denoted by w. The matching criterium generally consists in computing the mean absolute difference (MAD) between the blocks. Let x(2, 7) be the pixels of the reference block and y(i,7) the pixels of the candidate block. The matching distance (or distortion) D is computed according to (20). The indexes m and n indicate the position of the The algorithm for computing a motion vector can be sequentially expressed as follows (see bottom of following page). PIRSCH et al.: VLSI ARCHITECTURES FOR VIDEO COMPRESSION—A SURVEY
candidate block within the search window. The distortion D is computed for all the (2w + 1)’ possible positions of the candidate block within the search window in (21) and the block corresponding to the minimum distortion is used for prediction. The position of this block within the search window is represented by the motion vector v, see (21). a search window, but several other approaches have been reported in order to reduce the computation requirements by using an “intelligent” or “directed” search [26], [29], [30], [34], [36], [39], [40]. oe ae Ree ee eee eee oe 1) Exhaustive Search Block Matching: Applying an ex- haustive search block matching algorithm, a block of size Nx\N of the current image (reference block, denoted X) is matched with all the blocks located within a search window (candidate blocks, denoted Y). The maximum displacement will be denoted by w. The matching criterium generally consists in computing the mean absolute difference (MAD) between the blocks. Let x(2, 7) be the pixels of the reference block and y(i,7) the pixels of the candidate block. The matching distance (or distortion) D is computed according to (20). The indexes m and n indicate the position of the The algorithm for computing a motion vector can be sequentially expressed as follows (see bottom of following page). PIRSCH et al.: VLSI ARCHITECTURES FOR VIDEO COMPRESSION—A SURVEY
The dependence graph for computing D(m,n) can be mapped into a 1D array of Processing Elements, which is able to compute in paraliel the partial distortion along one row. This 1D array can sequentially compute D(m,7) in N cycles. The dependence graph for computing V(X, Y) is A direct mapping of the dependence graphs of Fig. 11 is possible. The dependence graph for computing D(m, 7) is directly mapped into a 2D array of processing ele- ments (PE), while the dependence graph for computing v(X,Y) is mapped into time (Fig. 13). In other words, block matching is performed by a sequential exploration of the search area, while the computation of each distortion is performed in parallel. Each of the AD nodes of the DG is implemented by an AD processing element (AD- PE). The AD-PE stores the value of x(¢,7) and receives the value of y(m + i, + 7) corresponding to the current position of the reference block in the search window. It performs the subtraction and the absolute value compu- tation, and adds the result to the partial result coming from the upper PE (Fig. 14). The partial results are added on columns and a linear array of adders performs the horizontal summation of the row sums, and computes D(m,n). For each position (n,m) of the reference block, the M-PE checks if the distortion D(m,n) is smaller than the previous smaller distortion value, and, in this case, updates the register which keeps the previous smaller distortion value.
The dependence graph for computing D(m,n) can be mapped into a 1D array of Processing Elements, which is able to compute in paraliel the partial distortion along one row. This 1D array can sequentially compute D(m,7) in N cycles. The dependence graph for computing V(X, Y) is A direct mapping of the dependence graphs of Fig. 11 is possible. The dependence graph for computing D(m, 7) is directly mapped into a 2D array of processing ele- ments (PE), while the dependence graph for computing v(X,Y) is mapped into time (Fig. 13). In other words, block matching is performed by a sequential exploration of the search area, while the computation of each distortion is performed in parallel. Each of the AD nodes of the DG is implemented by an AD processing element (AD- PE). The AD-PE stores the value of x(¢,7) and receives the value of y(m + i, + 7) corresponding to the current position of the reference block in the search window. It performs the subtraction and the absolute value compu- tation, and adds the result to the partial result coming from the upper PE (Fig. 14). The partial results are added on columns and a linear array of adders performs the horizontal summation of the row sums, and computes D(m,n). For each position (n,m) of the reference block, the M-PE checks if the distortion D(m,n) is smaller than the previous smaller distortion value, and, in this case, updates the register which keeps the previous smaller distortion value.
Fig. 10. Normalized silicon area and through-put for dedicated DCT circuits.
Fig. 10. Normalized silicon area and through-put for dedicated DCT circuits.
Fig. 11. Dependence graphs of the block matching algorithm. The computation of v(.X,Y) and D(m,n) are performed by 2D linear DG’s.
Fig. 11. Dependence graphs of the block matching algorithm. The computation of v(.X,Y) and D(m,n) are performed by 2D linear DG’s.
Fig. 12. Other possible DG’s for the computation of D(m,n). Due to the associativity of the operations, similar DG’s could be used for the computation of D(m, 7).
Fig. 12. Other possible DG’s for the computation of D(m,n). Due to the associativity of the operations, similar DG’s could be used for the computation of D(m, 7).
explore the reference block. At each cycle, one PE receives a different value of y(m + %,n + 7) and all the PE receive the value of one pixel of the reference block which is broadcasted to the array. After N cycles, each of the (2w+1) Processing Elements holds one value of D(m,n) corresponding to a particular displacement (m,n). These values can be stored in a slave register, which allows to start a new computation, while computing the minimum distortion value. One way to do this is to find the minimum value along the column by downshifting the D(m,n) in M-PE’s and find the final minimum value by left-shifting the resulting D(m,n) in the M-PE’s. This process takes (2w+1)* cycles, which, for commonly used values of w and N, is smaller than the N* cycles available to perform
explore the reference block. At each cycle, one PE receives a different value of y(m + %,n + 7) and all the PE receive the value of one pixel of the reference block which is broadcasted to the array. After N cycles, each of the (2w+1) Processing Elements holds one value of D(m,n) corresponding to a particular displacement (m,n). These values can be stored in a slave register, which allows to start a new computation, while computing the minimum distortion value. One way to do this is to find the minimum value along the column by downshifting the D(m,n) in M-PE’s and find the final minimum value by left-shifting the resulting D(m,n) in the M-PE’s. This process takes (2w+1)* cycles, which, for commonly used values of w and N, is smaller than the N* cycles available to perform
this computation. This architecture must also be extended, in a manner similar to the extension described in Fig. 15, for storing a part of the search area of size w -(2w +N), in order to reduce the I/O’s.
this computation. This architecture must also be extended, in a manner similar to the extension described in Fig. 15, for storing a part of the search area of size w -(2w +N), in order to reduce the I/O’s.
A 1D search-area based architecture can be designed, where an array of (2w + 1) processing elements executes in N? cycles the computation of the distortions D(m,n) corresponding to one line (resp. column) of possible motion vectors. This process is repeated sequentially 2w + 1 times for computing all the distortions. [51], [18], [86] have implemented such 1D arrays of 16 processing elements (Fig. 19).
A 1D search-area based architecture can be designed, where an array of (2w + 1) processing elements executes in N? cycles the computation of the distortions D(m,n) corresponding to one line (resp. column) of possible motion vectors. This process is repeated sequentially 2w + 1 times for computing all the distortions. [51], [18], [86] have implemented such 1D arrays of 16 processing elements (Fig. 19).
Typically, 1) is implemented by search strategies in suc- cessive steps. As an example, a modified scheme according to the original proposal of [36] will be discussed. In this scheme the best match v,_1 in the previous step s — 1 is improved in the present step s by comparison with displacements +A,. The displacement vector v, for each step s is calculated according to Ag, depends on the maximum displacement w and the number of search steps N,. Typically, when w = 2k—1, Ns is set to k = logo(w + 1) and D, = 2k — s + 1. For example, for w = 15, four steps with D, = 8, 4, 2, 1 are performed. This strategy reduces the number of candidate blocks from (2w + 1)? in case of exhaustive search to 1+8-log.(w+1), e.g., for w = 15 the number of candidate blocks is reduced from 961 to 33 which leads to a reduction of processing power by a factor of 29. For large block sizes N, the number of operations for the match can be further
Typically, 1) is implemented by search strategies in suc- cessive steps. As an example, a modified scheme according to the original proposal of [36] will be discussed. In this scheme the best match v,_1 in the previous step s — 1 is improved in the present step s by comparison with displacements +A,. The displacement vector v, for each step s is calculated according to Ag, depends on the maximum displacement w and the number of search steps N,. Typically, when w = 2k—1, Ns is set to k = logo(w + 1) and D, = 2k — s + 1. For example, for w = 15, four steps with D, = 8, 4, 2, 1 are performed. This strategy reduces the number of candidate blocks from (2w + 1)? in case of exhaustive search to 1+8-log.(w+1), e.g., for w = 15 the number of candidate blocks is reduced from 961 to 33 which leads to a reduction of processing power by a factor of 29. For large block sizes N, the number of operations for the match can be further
Fig. 19. 2D architecture of [51]. 2) Block Matching Algorithms With Reduced Computational Complexity: Exhaustive search block matching requires high computational power. For example for CCIR signals and w = 15 a processing power of approximately 30 giga operations per second (GOPS) is required.
Fig. 19. 2D architecture of [51]. 2) Block Matching Algorithms With Reduced Computational Complexity: Exhaustive search block matching requires high computational power. For example for CCIR signals and w = 15 a processing power of approximately 30 giga operations per second (GOPS) is required.
Table 5 Comparison of Reported Motion Estimation Circuits reduced by combining the search strategy with subsampling in the first steps. Architectures for block matching based on hierarchical search strategies are presented in [48], [44], [35]. [33], [24], [31].
Table 5 Comparison of Reported Motion Estimation Circuits reduced by combining the search strategy with subsampling in the first steps. Architectures for block matching based on hierarchical search strategies are presented in [48], [44], [35]. [33], [24], [31].
Fig. 20. Normalized silicon area and computational rate for ded- icated motion estimation architectures.
Fig. 20. Normalized silicon area and computational rate for ded- icated motion estimation architectures.
Fig. 21. MPEG-Decoder according to [84]
Fig. 21. MPEG-Decoder according to [84]
Fig. 22. MPEG-Decoder according to [62].
Fig. 22. MPEG-Decoder according to [62].
Fig. 24. VSP3 architecture [71]. Fig. 23. Increase of through-put rate by pipelining.
Fig. 24. VSP3 architecture [71]. Fig. 23. Increase of through-put rate by pipelining.
Fig. 25. IDSP architecture [91].
Fig. 25. IDSP architecture [91].
Fig. 26. Vision processor architecture [58].
Fig. 26. Vision processor architecture [58].
Fig. 27. Vision controller overview [58].
Fig. 27. Vision controller overview [58].
subtracter-complementer-multiplier pipeline. The datapath can be configured to two 16-b pipelines. In both cases, the intermediate results of the arithmetic pipelines are fed into a multi operand accumulator with shift/limit circuitry. The results of the APU can be stored in the internal local memory or read out to the external data output bus.
subtracter-complementer-multiplier pipeline. The datapath can be configured to two 16-b pipelines. In both cases, the intermediate results of the arithmetic pipelines are fed into a multi operand accumulator with shift/limit circuitry. The results of the APU can be stored in the internal local memory or read out to the external data output bus.
Fig. 32. TMS320C80 (MVP) [68].
Fig. 32. TMS320C80 (MVP) [68].
Table 6 Overview of Programmable Architectures for Video Coding Applications
Table 6 Overview of Programmable Architectures for Video Coding Applications
fig. 33. Normalized silicon area and throughput (frame rate) for programmable architectures for 1 H.261 codec.
fig. 33. Normalized silicon area and throughput (frame rate) for programmable architectures for 1 H.261 codec.
Related topics:
Biomedical EngineeringVideo CompressionProcessor ArchitectureBlock MatchingElectrical and Electronic EngineeringFigure of MeritVLSI architecture
580 California St., Suite 400
San Francisco, CA, 94104
© 2025 Academia. All rights reserved