Figure 4 – uploaded by Manoranjan Paul

See full PDF downloadDownload figure

Table 2.2: Exp-Golomb code Exponential Golomb (Exp-Golomb) [32] VLC code which is popular because its — Table 2 2: Exp-Golomb code Exponential Golomb (Exp-Golomb) [32] VLC code which is popular because its

Related Figures (77)

Figure 1.1: An example on how pattern-based coding can exploit the intra-block temporal redundancy in improving coding efficiency. already VLBR video coding option.

Figure 1.2: An example of pattern approximation for the Miss America standard video sequence, (a) frame number one, (b) frame number two, (c) detected moving regions, and (d) results of pattern approximation.

Figure 2.1: Generic block-based video encoder structure.

Figure 2.2: Generic block-based video decoder structure. DCT (IDCT) are used to form the reference frames which are stored in the frame memory for

Figure 2.3: A typical GOP for digital video coding.

Table 2.1: Encoding order and display order of frames in a typical GOP 2.2.1.2 Motion Estimation Techniques so the first P-frame must be processed before the first two B-frames.

Search window of a reference frame distance-dependent thresholding search proposed by Sorwar, Murshed, and Dooley [124]. Figure 2.4: Block motion vector estimation.

Figure 2.7: (a) A typical quantisation matrix, (b) DCT coefficients after applying scalar quantisation to Figure 2.6(b) where the value is 16, and (c) DCT coefficients after applying a quantisation matrix on Figure 2.6(b). small number of bits can represent a data, and thus compression is achieved. For compression, the DCT coefficients are quantised by either a scalar quantisation value o quantisation matrix. A quantisation matrix (see Figure 2.7(a)) typically has small values in th top-left elements of the matrix and larger values in the bottom-right, so retaining lov frequency information at the expense of higher frequencies (see the difference between Figur 2.7(b) and (c)). Using the quantisation matrix however, is very computationally expensivi especially when the coding control mechanism is used to adjust the bit rate. After applyins quantisation, the magnitude of data is reduced (see Figure 2.7(b)). As a result, a relatively emall number of hits can renresent a data and this camnrescion is achieved quantisation, the magnitude of data is reduced (see Figure 2.7(b)). As a result, a relatively

Figure 2.9: (a) Zigzag scanning order, (b) coefficients derived by zigzag scanning on Figure 2.7(b), and (c) variable length codes and zero run length codes from zigzag scanned data using the H.263 recommended VLC look-up table.

Table 2.3: Comparison of digital video coding standards MPEG-1 and MPEG-2 2.3 Block Based VLBR Video Coding

Figure 2.10: Relative frame size of CIF (largest), QCIF, and Sub-QCIF (smallest) format using the first frame of the Miss America video sequence as the example.

Table 2.4 Various video frame formats used in digital video coding standards While ISO MPEG video/audio standards focused upon generic coding applications, namely the Very Low Bit Rate Video Coding: A Review

Figure 2.11: Variable block sizes for the H.264 standard: (a) 16x16, 16x8, 8x16, and 8x8, (b) 8x8, 8x4, 4x8, and 4x4.

Figure 2.12: Graphical representation of 4x4 grey prediction modes [109]. 8, the predicted samples, are obtained from a weighted average of the prediction samples A-M. arrows in Figure 2.12 indicate the direction of prediction in each mode. For example, modes 3- The nine modes are as follows: i) mode 0 (vertical): where the upper samples A, B, C, and The nine modes are as follows: i) mode 0 (vertical): where the upper samples A, B, C, and D are extrapolated vertically; ii) mode 1 (horizontal): where the left samples L, J, K, and L are extrapolated; iii) mode 2 (DC): where all samples in the predicted block are obtained by the mean of samples A...D and I...L; iv) mode 3 (diagonal down-left): where the samples are interpolated at a 45° angle between lower-left and upper-right; v) mode 4 (diagonal down- right): where the samples are extrapolated at a 45° angle down and to the right; vi) mode 5 (vertical-right): where there is extrapolation at an angle of approximately 26.6" to the left of the vertical (width/height = 1/2); vii) mode 6 (horizontal-down): where there is extrapolation at an angle of approximately 26.6° below the horizontal; viii) mode 7 (vertical-left): where there is extrapolation (or interpolation) at an angle of approximately 26.6° to the right of the vertical; ix) mode 8 (horizontal-up): where there is interpolation at an angle of approximately 26.6° above the horizontal.

Table 2.6: Quantisation step size used in the H.264 CODEC The VLBR coding especially in videophone applications, it is common for the coefficients of a block to be zero after motion estimation/compensation, DCT and quantisation. A method

Figure 2.13: Vector Quantisation coder and decoder. The set Y is called the codebook and each element of Y is called a codeword. Figure 2.13 shows the functional block diagram of a basic VQ video coder (blocks shown

Figure 2.14: Basic schematic diagram of model-based image coding. In image modelling [3][125][17][11][142], a geometric wireframe model is used for the

Figure 2.15: Wire-frame model of the face: (a) front view, (b) profile, and (c) synthesized face image assuming simple surface and one light source [103], [81]. advantages of MBC and waveform MC-DCT to overcome the limitations of MBC. Although MBC opens the possibility of very low bit rate image transmission, the

Figure 2.18: Block diagram of pattern-based video coding encoder.

Figure 2.19: Block diagram of pattern-based video coding decoder. Very Low Bit Rate Video Coding: A Review

Figure 2.20: Rearrangement of the residual errors so that only one 8x8 block is needed to accommodate the 64 pixels position in a pattern [141]. patterns do not need rearranging because they are already in one 8x8. Since DCT is applied on

Figure 2.21: Standard QCIF video sequences with their motion type, (a) Miss America (“talking heads” motion), (b) Suzie (hand and head movements), (c) MotherDaughter (two heads movements), (d) Carphone (fast object motion), (e) Foreman (object translation and panning), (f) Salesman (head- shoulder-hand movements), and (g) Claire (“talking head” motion). example, Carphone possesses relatively high background object motion compared with either

Figure 3.1: Percentage of different size blocks (a) and sub-blocks (b) selected by H.264 at different bit rates for the Salesman sequence.

Figure 3.2: (a) Frame number two of the Miss America sequence, (b) different size blocks, and (c) and (d) different size blocks of frame number two at low and high bit rate respectively.

Figure 3.3: Example MB (a) with quadrants, (b) with few moving pixels in I and II quadrants, and (c) with huge moving pixels in I, II, and IV quadrants. 3.3(c) is unnecessarily matched with the pattern codebook as there is no ‘1’ in quadrant II.

Table 3.1: Actual number and percentage of different MBs generated by the Fixed-8 algorithm using the quadrant based MB classification The rule for classifying the type of the k-th MB is defined as:

Figure 3.4: Increase in the number of RMBs (%) by the Fixed-8 algorithm with parametric MB classification compared to the quadrant based classification.

Figure 3.5: Number of RMBs generated by the Fixed-8 algorithm applied on the first 100 frames of the Miss America video sequence for different 5 values in parametric MB classification.

parametric MB classification, unless stated otherwise. Figure 3.6: Percentage of increased number of RMBs vs. pattern codebook sizes for seven standard video sequences.

Figure 3.7: The extended pattern codebook of 32 regular shaped, 64-pixel patterns, defined in 16x16 blocks, where the white region represents ‘1’ (motion) and the black region represents ‘0’ (no motion). shown in Figure 3.7, where the first eight represents the patterns used by Wong et al. regularity feature is added to limit the pattern codebook size. The full 32-pattern codebook is

Figure 3.8: Percentage of increased RMBs generated by the Fixed-32 algorithms compared to the Fixed- 8 algorithm using parametric MB classification with O= 120.

Table 3.2: Pattern numbers with their corresponding RMB frequency (%) and VLC using a large number of standard and non-standard video sequences

Figure 3.9: Change in the frequency of RMB captured by each pattern while the second best similar pattern is also considered along with the best similar pattern and the one leading to higher overall compression is selected.

Table 3.3: Pattern number with their corresponding RMB frequency (%) and VLC using a large number of standard and non-standard video sequences where the second best similar pattern might replace the best similar pattern in favour of overall coding efficiency and the most frequent pattern has a null code

Table 3.4: The average length of each pattern identification code using the VLC (Huffman) and the enhance VLC schemes Figure 3.10 shows the comparative coding performance using the Fixed-8 and the Fixed- 32 algorithms for the Miss America sequence. The figure confirms that the PVC algorithm

Figure 3.10: Coding efficiency curve for the Miss America video sequence using the Fixed-8, Fixed-32, Fixed-32 with VLC (Fixed-32_VLC), Fixed-32 with VLC and new parametric MB classification (Fixed-32_VLC_PMB), and the Fixed-32 with EVLC and new parametric MB classification (Fixed- 32_EVLC_PMB). fraction of bit per RMB.

la Ab), = lal, - la A |, where a and b are Boolean variables Lemma 3.1: |a ~ 4], =\a|, +|b], -2|a\d|, where a and b are Boolean variables. Table 3.5: Truth table proof of relations la ~ d, = la|, + \D|, - 2\a Ad, an

Figure 3.11: (a) Similarity example for a moving region M of a CRMB and pattern P 1; (b) two non- overlapping areas (black) relevant to the PISM; (c) non-overlapping area (black) relevant to the PESM. from a set of CRMBs: heuristic is therefore justified in order to classify the approximately similar number of RMBs

Table 3.6: Percentage of RMBs with respect to the total MBs and average moving region size of RMBs generated by the Fixed-32 algorithm using PISM and PESM * The term PISM (also PESM) refers to both a similarity metric, and the Fixed-32 algorithm using the same similarity metric, that are readily identifiable based on the context. Theorem 3.1: PISM and PESM are equivalent in identifying the best pattern for any moving

Enhanced Pattern Based Video Coding

Figure 3.12: Flowchart of an efficient computational strategy on (a) the PISM, (b) the PESM, where NOP means no operation. regard to whether an MB is classified as an RMB or AMB.

Table 3.7: Comparative computational complexity of the Fixed-32 algorithm using the PISM and PESM against H.264* using (3.6)-(3.8) with the percentages of CRMBs, and AMBs

Enhanced Pattern Based Video Coding

Figure 3.13: Coding efficiency curves for the standard video sequences using the Fixed-32 (with new parametric MB classification, efficient VLC, and PESM), Fixed-8, and the H.264*. Enhanced Pattern Based Video Coding

Figure 3.14: Coding efficiency curves for the standard video sequences using the Fixed-32 (with efficient VLC, parametric MB classification, and PESM), Fixed-8, and the H.264*.

Figure 3.15: Frame level PSNR in the Miss America sequence using the H.264*, Fixed-8, and the Fixed-32 algorithms for target bit-rate 22 Kbps. Enhanced Pattern Based Video Coding

Figure 3.16: (a) Miss America frame number three, (b)—(d) reconstructed frames using the H.264*, Fixed-8, and the Fixed-32 algorithms respectively, (e)-(g) frame differences (x3) of (b), (c), and (d) respectively with respect to (a). Enhanced Pattern Based Video Coding

Figure 3.17: (a) Claire frame number three, (b)—(d) reconstructed frames using the H.264*, Fixed-8, and the Fixed-32 algorithms respectively, (e)-(g) frame differences (<3) of (b), (c), and (d) respectively with respect to (a). Enhanced Pattern Based Video Coding

Table 4.1: Relevance threshold values Tr(7min) fora dynamically created CPC having a lower bound of min and an upper bound of 7max (77min) patterns size of a CPC. As with Tr(7min), because of the infinite number of possible real coordinates,

controlling the number of patterns for each CRMB.

Figure 4.2: Speed-up factor, 17min ) for different minimum number of patterns, 77min on the Miss America sequence.

Figure 4.5: Speed-up factor (11) and (32) on seven standard sequences. Lemma 4.5: Fixed-32 algorithm requires 256+(M +33)A “add-equivalent” operations. be proven by means of a similar argument to that used in [141].

Figure 4.7: Percentage of RMBs where the same pattern is selected by the RTPS(11) & Fixed-32, RTPS(4) & Fixed-32, and the Fixed-8 & Fixed-32 algorithms.

Figure 4.8: Percentage of RMBs in the Miss America sequence where the best similar pattern (captured by the Fixed-32 algorithm) is selected by the RTPS(7min) algorithm using the proposed iso-7Tz technique and the straightforward technique of selecting of the first 77a,2(77min) best relevant patterns. sorting for lower values of 77min. straightforward selection of the first 7ave(77min) best relevant patterns in the CPC through

Table 4.2: PSNR values for standard sequences using the RTPS(2) and the RTPS(11) algorithms. Although the RTPS algorithm allows us to use any arbitrary large size of pattern codebook

Figure 5.1: 64 patterns generated using definition where white indicates the moving region and black indicates the static region. while edges of the patterns are aligned to the block boundaries as much as possible. in Figure 5.1 are considered so that the inter-relations among them are kept to a minimum

Figure 5.2: Patterns generated using (a) Euclidean distance, (b) Manhattan distance, and (c) Chebyshev distance.

Figure 5.3: RMB frequencies of the best eight patterns among 64 patterns shown in Figure 5.1. this sense, definition based pattern generation would, ultimately, prove a good solution.

Figure 5.6: Actual and approximated (dotted lines) values of iterations of EVPS(/ € {4,8,16,32}, ye {51%,...,100%}) algorithm for the Suzie sequence. Efficient Pattern Codebook Downsizing

Figure 5.7: The intermediate pattern set size of the EVPS(4, 75%) algorithm for the Suzie sequence where doted line is assumed as intermediate pattern set size for calculation. VLC of pattern numbers. Considering a-4A selection iterations and Q,| =a-i+l1, the VPS

Figure 5.7 shows the example of 1 = 4 as it represents the worst case of pattern elimination.

Table 5.1: Percentage of RMBs for standard sequences using the EVPS(32,75%), Fixed-32, Fixed-|3- Opt(8)|, EVPS(8,75%), and the Fixed-8 algorithms with respect to the Fixed-|3-Opt(32)| algorithm

Figure 6.2: (a) 3-D representation of pixel frequency of one of the eight clusters of CRMBs obtained for Miss America sequence; (b) its corresponding 2-D top view projection; and (c) the generated pattern for this cluster by the CPG algorithm. represents only local optima. pattern is constrained to represent the entire cluster. However, such optimality, in reality, pattern generated by the CPG algorithm guarantees an optimal similarity measure if only on

Figure 6.3: 64-pixel size patterns extracted from video sequences by the CPG algorithm. patterns. one of the main reasons for the superior coding efficiency achieved by the arbitrary shaped

Figure 6.4: Percentages of RMBs in a cluster using the pattern generated by the same cluster in APVC algorithm for different number of processing frames where PC size is eight.

Figure 6.5: Performance of APVC algorithm with different PC sizes where each pattern has 64-pixel. omputational complexity in calculating the similarity metric calculation due to the larg

Figure 6.6: Performance of the APVC algorithm using different similarity thresholds for the Miss America sequence with eight patterns.

Table 6.1: Percentage of different MB types (defined in Section 3.3) generated by the APVC algorithm 6.3 The Scalable APVC (SAPVC) Algorithm

Table 6.2: Rules for classifying MB types in SAPVC MBs are classified into five types in pattern generation phase in SAPVC such as, SMB Arbitrary Shaped Pattern Based Video Coding

Figure 6.8: Percentage of MB types generated by the SAPVC-2 algorithms for different standard video sequences. Arbitrary Shaped Pattern Based Video Coding

Figure 6.9: Coding performance comparisons for standard test video sequences using the SAPVC-2, the SAPVC-1, the APVC, the RTPS(11), and the H.264* standard. Arbitrary Shaped Pattern Based Video Coding

Figure 6.10: Coding performance comparisons for standard test video sequences using the SAPVC-2, the SAPVC-1, the APVC, RTPS(11), and the H.264* standard. those sequences have relatively large moving regions which are well represented by large

Figure 6.11: (a) MotherDaughter frame number 13, (b)—(e) reconstructed frames using the H.264*, Fixed-8, RTPS(11), and the APVC respectively, (f)—(i) frame differences (x2) of (b), (c), (d), and (e) respectively with respect to (a). Arbitrary Shaped Pattern Based Video Coding