Video Coding Research Papers

Compressed domain content based retrieval using H.264 DC-pictures

2012, Multimedia Tools and Applications

A fast and simple method for content based retrieval using the DC-pictures of H.264 coded video without full decompression is presented. Compressed domain retrieval is very desirable for content analysis and retrieval of compressed image... more

descriptionView Paper arrow_downwardDownload

Fovea Based Coding for Video Streaming

by Reha Civanlar

2004, Lecture Notes in Computer Science

descriptionView Paper arrow_downwardDownload

Study of High Level design methodologies for a MPEG frames I Compressor directed to FPGA Implementation

by Antoni Portero

2004, cephis.uab.es

Time needed to set-up and send new products to the market is a crucial factor for success, so one important rule is to develop your product as quickly as you can. In the electronics world the systems normally contain some parts in HW and... more

descriptionView Paper arrow_downwardDownload

R-D Spatio-Temporal Adaptive Quantization based on Temporal Distortion Backpropagation in HEVC

by Julien Le Tanou and

Nowadays, Rate-Distortion Optimization (RDO) is commonly used in hybrid video coding to maximize coding efficiency. Usually, the rate distortion tradeoff is explicitly computed in offline encoder implementations whereas R(D) model are... more

One “next” (in coding order) CU is partially impacted (according to its motion prediction information and its INTER probability) by the distortion produced by the “current” CU. Generalizing the propagation to several images leads to successively accumulate distortions along the GOP, such the total distortion formula D,,; is: We then introduce the derivative of the total distortion with the hypothesis that the distortion attached to a spatial position only depends on its local quantizer and that the bits produced (rate) only depend on its local quantizer. After some mathematical development, we get:

CODING EFFICIENCY OF ADAPTIVE QUANTIZATION MODELS OVER NO LOCAL QUANTIZATION IN X265

descriptionView Paper arrow_downwardDownload

A systems engineering perspective of aspect-oriented software architectural analysis using UML

by Jesus Rivera

2003

Literature on Unified Modeling Language (UML) and aspect-orientation often focus on improving the software development process [2]. This paper presents a systems engineering perspective that motivates an aspect-oriented software... more

descriptionView Paper arrow_downwardDownload

Using graphics rendering contexts to enhance the real-time video coding for mobile cloud gaming

by Roy Campbel

2011

The emerging cloud gaming service has been growing rapidly, but not yet able to reach mobile customers due to many limitations, such as bandwidth and latency. We introduce a 3D image warping assisted real-time video coding method that can... more

descriptionView Paper arrow_downwardDownload

Video streaming in mobile ad hoc networks

by Thomas Stockhammer

2006, Proceedings of the 2nd international conference on Mobile multimedia communications - MobiMedia '06

... 5: Scalable Video Coding (in integrated form with ITU-T Rec. H.264 | ISO/IEC 14996-10). JointVideo Team (JVT), Doc. JVT-R201, Jan. 2006. 4. Luby, M., Stockhammer, T., Watson, M., Gasiba, T., and Xu, W. Raptor Codes for Reliable... more

descriptionView Paper

Overview of the Scalable Video Coding Extension of the H.264/AVC Standard

by Detlev Marpe

2000, IEEE Transactions on Circuits and Systems for Video Technology

descriptionView Paper arrow_downwardDownload

Performance Analysis of SVC

by Heiko Schwarz

2000, IEEE Transactions on Circuits and Systems for Video Technology

This paper provides a performance analysis of the Scalable Video Coding (SVC) extension of H.264/AVC. A short overview presenting the main functionalities of SVC is given and main issues in encoder control and bit stream extraction are... more

Fig. 1. Coding structure example with two spatial layer.

Fig. 2. Hierarchical coding structures for providing temporal scalability. (a) Hierarchical B-pictures. (b) Low-delay coding structure.

Fig. 4. Comparison of FGS and MGS rate-distortion performance. Fig. 3. Key picture concept for hierarchical prediction structures. Between two key pictures (K), the enhancement representation (EL) is used for prediction. For the key pictures, the base representation (BL) is used.

Fig. 5. Test sequence Foreman. (a) QCIF 15 Hz. (b) CIF at 30 Hz. Rate-dis- tortion performance of SVC using quality and spatial scalability to H.264/AVC single layer coding and MPEG-4 ASP. H.264/AVC simulcast for the spatial scalability scenario is additionally shown.

Fig. 6. Test sequence Crew. (a) QCIF 15 Hz. (b) CIF at 30 Hz. (c) 4CIF 60 Hz. Rate-distortion performance of SVC using quality and spatial scalability to H.264/AVC single layer coding and MPEG-4 ASP. In (b), H.264/AVC simulcast for the spatial scalability scenario is additionally shown.

Fig. 7. Test sequence Crew. Results for the ESS test case. The rate of the base layer is as follows: 396 kbps (1/2), 558 kbps (3/4), and 617 kbps (2/3). The resulting PSNR value at the base layer is 36.11 dB for the 1/2 case, 36.94 dB for the 3/4 test case and 36.52 dB for the 2/3 test case.

Fig. 8. Rate-distortion efficiency improvement with an optimized multilayer encoder control relative to the JSVM for quality scalable coding of the sequence Soccer in CIF resolution with a frame rate of 30 Hz.

in electrical Cneimeecrming Mm i770 ana We VY.-ing. degree in 2000, both from the University of Rostock, Rostock, Germany. In 1999, he joined the Fraunhofer Institute for Telecommunications—Heinrich Hertz Institute (AHI), Berlin, Germany. Since then, he has con- tributed successfully to the standardization activities of the ITU-T Video Coding Experts Group (ITU-T SG16/Q.6—VCEG) and the ISO/IEC Moving Pictures Experts Group (ISO/IEC JTC 1/SC 29/WG 11—MPEG). During the development of the Scalable Video Coding (SVC) extension of H.264/AVC, he co-chaired several ad hoc groups of the Joint Video Team of ITU-T VCEG and ISO/IEC MPEG investigating particular aspects of the scalable video coding design. He has also been appointed as a Co-Editor of the SVC Amendment for H.264/AVC.

descriptionView Paper arrow_downwardDownload

Video Compression Codecs: A Survival Guide

by Iain Richardson

2017, IASA journal No 47

Since the first commercially viable video codec formats appeared in the early 1990s, we have seen the emergence of a plethora of compressed digital video formats, from MPEG-1 and MPEG-2 to recent codecs such as HEVC and VP9. Each new... more

Figure 3 Encoding and decoding steps The output of all these steps is a compressed bitstream - a series of binary digits (bits) that takes up much less space than the original video frames and is suitable for transmission or storage.

Figure 4 What a video coding standard covers Typically, a video coding standard does not define an encoder (Figure 4). It is up to each manu- facturer to decide how to design an encoder. The only requirement is that the bitstream pro- duced by the encoder must be compliant with the standard, i.e. it has to conform to the format described by the standard and it must be decodeable by the method defined in the standard.

Figure 5 Timeline of selected video coding standards Since the first digital video coding standards were developed in the late 1980s/early 1990s, there have been a surprising number of standards released. Figure 5 shows some (but not all) of the key standards released over the last 25 years. Many were developed by working groups of the ISO/IEC and ITU-T international standards organisations. ISO/IEC standards include MPEG-2 (ISO/IEC 13818-2 and ITU-T Recommendation H.262, 1995), the standard used for the first digital TV services and for DVD Video. Some standards have been co-published by ISO/IEC and ITU-T, which is why H.264 (ITU-T Recommendation H.264, 2003) (for example) is also known as MPEG-4 Part 10.The most recent publication of the ISO/IEC and ITU-T is H.265 / HEVC, High Efficiency Video Coding, first released in 2013 (ITU-T Recommendation H.265, 2013).

Figure 6 Container file There is considerable flexibility in the choice of container format, video format and audio for- mat. For example, an MP4 file can contain video formats such as MPEG-2, MPEG-4, H.264 or H.265 and audio formats including MP3, AAC and others.

Figure 7 Single encode and decode It is important to be aware that each transcoding step - more specifically, each encoding step - can introduce quality loss into the video and/or audio content. If the encoding step is lossy, then degradation is introduced every time the content is re-coded. This may lead to generation loss (Figure 9), such that the visual quality of the material progressively degrades with each conversion.

Figure 10 Adaptive streaming DASH and other adaptive streaming technologies require multiple transcoded versions of the video clip to be created, with each section stored in a container file so that the required switchover points are available.

* Sometimes referred to as 4K, although the Digital Cinema Initiative 4K specification contains 4096x2 | 60 pixels per frame. Figure | compares the resolution of popular video formats:

descriptionView Paper arrow_downwardDownload

AI-BASED MEDIA CODING AND BEYOND

by Alessandro Artusi and

2021, International Broadcasting Convention IBC

MPAI-Moving Picture, Audio and Data Coding by Artificial Intelligence is the first body developing data coding standards that have Artificial Intelligence (AI) as its core technology. MPAI believes that universally accessible standards... more

descriptionView Paper arrow_downwardDownload

A rate control technique for off line H.264/AVC video coding using subjective quality of video

by Steven Adedoyin

2008, IEEE Transactions on Consumer Electronics

In this paper, we propose a rate control technique for H.264/AVC using subjective quality of video for off line video coding. We propose to use Video Quality Metric (VQM) with an evolution strategy algorithm, which is capable of... more

descriptionView Paper arrow_downwardDownload

Image and video coding-emerging standards and beyond

by Atul Puri and

1998, IEEE Transactions on Circuits and Systems for Video Technology

In this paper, we make a short foray through coding standards for still images and motion video. We first briefly discuss standards already in use, including: Group 3 and Group 4 for bilevel fax images; JPEG for still color images; and... more

descriptionView Paper arrow_downwardDownload

Wireless CDN video streaming architecture for IPTV

by Benjamin Molina

2011, Multimedia Tools and Applications

Wireless IPTV (Internet Protocol TeleVision) aims to make the traditional IPTV and related services available to users anywhere, anytime, on any device, and through any network. Mobile streaming TV is one of the alternatives to deploy... more

descriptionView Paper arrow_downwardDownload

Bit-Depth Scalable Video Coding

by Detlev Marpe

2007, 2007 IEEE International Conference on Image Processing

Scalable video coding (SVC) is an extension of H.264/AVC, which is currently being developed by the Joint Video Team. SVC supports any combination of temporal, spatial, and SNR scalability, but there is no special treatment for the case... more

Fig. 1: Basic architecture of our approach. case, decoding is still possible with only one single predic- tion loop.

Fig. 2: Example for different luma tone mapping schemes (left: linear, right: according to [5]).

Fig. 3: Typical inverse luma tone mapping characteris tic. 8 bit version of the sequence using a single-layer encoder where the reconstructed 8 bit signal has been inverse tone mapped to obtain a 10 bit signal using the same mapping scheme that has been employed for the scalable encoding process. The resulting plots are shown in Fig. 4. Note that the PSNR values are measured in the 10 bit sample domain. It can be seen that our scalable approach (red curves) clearly outperforms the variant using 8 bit data only (green curves) and nearly reaches the performance of a 10 bit single-layer coder (black curves) for the case using the rather complex tone mapping operator described in [5]. For the case of a linear tone mapping, as done for the Plane sequence, no gains to using the upscaled 8 bit base layer can be obtained by our scalable approach. For a further comparison, we also performed a simulation with a different inverse tone map- ping scheme. Here, only a linear scaling of the recon- structed 8 bit base layer signal was allowed where the scal- ing factor was independently chosen for each 16x16 block of the sequence with an accuracy of one half. The resulting rate distortion performance is shown in the blue curves. It can be seen that using only a linear scaling operation is not sufficient in the case that a more complex tone mapping operator than linear scaling is used.

Fig. 4: Rate distortion performance of our approact using three different test sequences.

descriptionView Paper arrow_downwardDownload

مروری بر قابلیت‌های تبدیلات فراکتالی برای فشرده‌سازی تصاویر رقمی

by Aso Shojaie

جهانی که در آن زندگی می‌کنیم، سرشار است از اشکال پیچیده‌ای که با قوانین هندسه اقلیدسی نمی‌توان آن‌ها را توصیف کرد. سلول‌های بدن‌مان، سیستم گردش خون، درختان، کوه‌ها، ابرها، رودها و تمامی آنچه طبیعت نامیده‌ایم، به زبان ریاضیات زیبایی به نام... more

descriptionView Paper arrow_downwardDownload

A Random Access Protocol for Configurable Video Coding

by Iain Richardson

2009, ISO/IEC JTC1/SC29/WG11 MPEG2009 M16927

We propose a random access protocol (RAP) for a configurable video coding framework. This protocol provides the data structure and stream for the transport of configuration data as well as mechanisms for achieving random access and... more

Figure 3: Header structure of each data unit type. Non-sequential unit streams can be used depending of the underlying transport system and synchronisation method used, as will be discussed in the following section. Figure 2. A sequential unit stream for fully configurable video coding.

Table 1: Example decoder configuration data sizes. We first address the issue of overhead size. In order to illustrate the overhead needed to transmit various decoder functionalities, Table 1 summarises decoder configuration data sizes for individual typical video decoder functions, the inverse discrete cosine transform (DCT) and the inverse Haar transform, a simple codec example an intra decoder, and a full H.264 decoder (JM 12.1) by using the decoder description syntax defined in *. Both the original decoder configuration data size and the size after being compressed by an entropy encoder are presented. A greater than 50% reduction in size is achieved by entropy coding. We first address the issue of overhead size. In order to illustrate the overhead needed When applying configurable video coding in broadcast scenarios, channel changing becomes more complicated as it involves immediate access to both video data and decoder configuration descriptions. Sufficient configuration information needs to be received for generating the up-to-date decoder to decode the video data starting from the RAP. When designing the tools for random access, two issues must be taken into consideration: (1) the overhead of transmitting decoder configuration information at each RAP need to be minimised to maintain good transmission efficiency compared with conventional video coding; and, (2) whether additional delay of receiving configuration data and creating decoder is negligible for channel zapping.

Table 2: Example of decoder configuration updates. 6. Simulation results A search algorithm can be used to eliminate any redundant updates. Redundant updates exist in the following circumstances: a function has been created more than once and therefore older versions are regarded as redundant; and, a function has been deleted prior to the requested RAP and any previous versions are redundant (c) a function has been modified and created again so that the modification should be treated as redundant. After reduction of redundant updates, sufficient decoder configuration information for the requested random access point will be sent to the receiver with video. This method will increase the transmission efficiency by not sending any unnecessary decoder configuration updates.

Table 4: Overhead of various configuration scenarios for direct broadcast and video streaming applications.

Table 6: Delay of for decoder configuration data transmission and decoder reconfiguration for broadcast and video streaming applications. 7. Conclusions We propose a protocol for enabling random access into a compressed video stream when decoder reconfiguration is transmitted along with the compressed video data. Since the reconfiguration information is tied to particular frames in the coded sequence, careful synchronization must be maintained between the decoder Sonfiguration data and the compressed video stream. To achieve the goal of efficient ‘random access, we have proposed separating the compressed video data and the Sompressed decoder configuration data into units and either sequentially transmitting he units in coding groups, or using identification codes when non-sequential ransmission is desired. 3.11ms for an Intra decoder when a basic decoder is in use and it is re-configured to be an intra decoder. When the same intra decoder is used in video streaming at 322kbps, the introduced delay is about 42.37ms. At the beginning of streaming, it takes about 1.76s to transmit an entire H.264 decoder, provided no basic decoder functions already exist at the decoder.

descriptionView Paper arrow_downwardDownload

Motion adaptive intra refresh for the H.264 video coding standard

by Albrecht Rothermel

2006, IEEE Transactions on Consumer Electronics

An intra refresh algorithm is presented which enables low-delay video communication at the efficiency of standard IP-coding schemes. It guarantees a complete recovery of erroneous transmission within a defined time interval at low... more

TEQCAriins Fig. 1 Frame data rate for test sequence “mobile & calendar” (H.264, q=35, 1 reference frame). At the beginning of the “intra refresh” video sequence a single I-frame with coarse quantization (q=51) is transmitted.

Intra refresh is a common technique to avoid large intra coded frames by embedding intra coded frame sections into a video stream comprising only P-frames. In previous work, several intra refresh techniques for applications in the field of video communications and low bit rate coding were proposed [1]. However, techniques requiring signaling from the receiver to the transmitter [2] are generally not suitable for low-delay video applications due to unacceptable round-trip times. Prediction confinement techniques [1] as well as the adaptive refresh [3] and random refresh [4] try to minimize possible error propagation by different means, but only the forced intra refresh method described in [5,6] can guarantee error-free recovery within a defined refresh time. Fig. 2 Principle of a region-based intra refresh (N = 4).

Fig. 3. Intra-Refresh border protection. The critical chequered border blocks are frequently intra-coded. Arrows represent motion vectors for temporal prediction. At the beginning of a cycle, all regions are marked as potentially corrupted “dirty” regions, which become “clean” regions when they are intra-coded in a frame. For ensuring perfect recovery within one refresh cycle, it is essential to avoid prediction from dirty regions of old reference frames into clean regions of the current frame. Otherwise, possible errors could propagate into the clean regions and damage the intra update. The forced intra refresh can guarantee a worst- case error-recovery within 2*N-1 frames. As a side-effect, the beginning of a refresh cycle can be used as a decoding start point like I-frames in IP streams.

The refresh sequence for the example in Fig. 4 is (1,2,3,4). Unprotected ,.dirty“ regions are marked gray.

computational complexity of the proposed algorithm, we defined a set of 28 strategies shown in Fig. 5. The sequences are related to common camera movement scenarios like pan, tilt and zoom. The set may be redundant or incomplete, but it worked well with all video sequences we evaluated. Fig. 5 Set of 4 * 3 region refresh patterns used for the simulations. The refresh starts with the region marked with a square and proceeds in the order indicated by the line.

Fig. 6 Simulation results for the proposed intra refresh algorithm sorted by descending motion activity. Estimation results are based on current motion and scaled to the measures range.

Fig. 7 Average region motion vectors calculated by the simulation. The length of motion vectors is adaptively scaled per sequence for better visibility.

descriptionView Paper arrow_downwardDownload

Processor Architectures for Multimedia Applications

by Peter Pirsch

2002, Lecture Notes in Computer Science

This textbook is intended to give an introduction to and an overview of stateof-the-art techniques in the design of complex embedded systems.

descriptionView Paper arrow_downwardDownload

Customization of an embedded RISC CPU with SIMD extensions for video encoding: A case study

by D. Reisis

2008, Integration, the VLSI Journal

This work presents a detailed case study in customizing a configurable, extensible, 32-bit RISC processor with vector/SIMD instruction extensions for the efficient execution of block-based video-coding algorithms utilizing a proprietary... more

Fig. 15. High-speed local memory configuration.

Fig. 6. High-level view of system. Fig. 5. Vectorized FSME and fast ME performance relative to scalar FSME (Set 1).

Fig. 7. Processor/coprocessor microarchitecture.

Fig. 13. Algorithm execution-time Data Cache study.

Fig. 14. Vector data Cache configuration.

Fig. 9. Vector register file (dual-port SRAM based). Fig. 10. Latch-based register file pipeline.

Fig. 12. Scalar registers and update logic.

Fig. 4. Vectorized MPEG2 TMS instruction count reduction (FSME).

Fig. 16. Coprocessor control pipeline schematic. To seamlessly integrate the coprocessor ISA to the existing Sparc V8 ISA, the unimp opcode was redefined as our study found that it is not produced by our Spare compiler. The decoding logic of the scalar CPU was modified such as not to generate an invalid instruction exception when the hijacked opcode was encountered. In addition, a number of control fields are generated within the RISC processor to enable the communication of scalar (32-bit) values across the processor—coprocessor interface. Another issue of great importance for the performance of a pipelined processor is the proper bypassing of operands flowing down the processor pipeline stages to the instruc- tion currently being decoded. Due to the adoption of a normally-illegal opcode, both the decoding and the bypassing logic itself had to be modified in order to enable

Fig. 17. Processor—Coprocessor VLSI layout of RAM-based VRF configuration with functional block overlay.

Fig. 18. Synthesis/power estimation campaign of the parametric vector processor.

VLSU performance (clocks/per/operation) Table 2 support for the vector datapath however, does not include un-aligned vector load/store operations as is the case with the RTL (Leon-2) model. A further point contributing to the better performance of the C-based simulation is the non-blocking Load/Store unit which, in the case of the Leon-2 model, uses a blocking configuration (sub-block misses during un-aligned vector accesses are a major source of stall cycles as two sub-blocks are fetched from memory). This is further exaggerated by the coherency logic of the VLSU which has extended the latency of a refill request appearing on the AHB. Finally, though the Simplescalar model was configured for single-issue/in-order operation it still models a more aggressive pipeline than the standard Leon-2 5-stage pipeline. ia. a

VLSI Macrocell results Vector processo rimplementation campaign

descriptionView Paper arrow_downwardDownload

Face detection: A survey

by Tran Viet

2001, Computer vision and image understanding

In this paper we present a comprehensive and critical survey of face detection algorithms. Face detection is a necessary first-step in face recognition systems, with the purpose of localizing and extracting the face region from the... more

FIG. 1. Typical training images for face recognition.

FIG. 2. A realistic face detection scenario.

FIG.4. Original image and M, of original image Reisfeld et al. [153].

FIG.5. Theface detection system of Maio and Maltoni [120]. Reprinted from Pattern Recognition, 33 Maio, D. and Maltoni, D., Realtime face location on gray-scale static images, 1525-1539, Copyright 2000, with permission from Elsevier Science. In the system of Maio and Maltoni [120], the input images are converted to a directional image using a gradient-type operator over local windows (7 x 7 pixels) (see also Froba and Kublebeck [47]). From this directional image they apply a two stage face detection method consisting of a generalized Hough transform and a set of 12 binary templates representing face constellations. The generalized Hough transform is used to generate face candidates by searching for ellipses. The face candidates are then fed to the template matching stage which accepts or rejects the face candidate as shown in Fig. 5. By efficient implementations and

FIG.6. Some examples of eigenfaces computed from the ORL dataset mentioned earlier (the number below each image indicates the principal component number, ordered according to eigenvalues). Let D =[®,,---®,] andC = DD'. The eigenvectors u; of C are called the principal components of D, and when converted back to matrices we can view these vectors as the eigenfaces of the dataset. Some examples of these eigenfaces are shown in Fig. 6 (ordered according to eigenvalue). These eigenfaces span the subspace called face space.

FIG.7. Classification in Sung and Poggio’s [182] system (© 1998/2001 IEEE). PCA is an intuitive and appropriate way of constructing a subspace for representing an object class in many cases. However, for modeling the manifold of face images, PCA is not necessarily optimal. Face space might be better represented by dividing it into subclasses, and several methods have been proposed for this, of which most are based on some mixture of multidimensional Gaussians. This technique was first applied for face detection by Sung and Poggio [182]. Their method consists mainly of four steps (Fig. 7):

FIG.8. The system of Rowley et al. [158] (© 1998/2001 IEEE).

FIG. 10. Face detection examples from Schneiderman and Kanade [170] (© 2000/2001 IEEE).

Average Lengths (Times the Reference Length) of Facial Features in De Silva et al.’s Algorithm TABLE 1

Results Reported in Terms of Percentage C orrect Detection (CD) and Numbe of False Positives (FP), CD/FP, on the CMU and MIT Datasets

descriptionView Paper arrow_downwardDownload

June 2022: Top 10 Read Articles in Signal & Image Processing

by Signal & Image Processing : An International Journal (SIPIJ) - WJCI Indexed

2022, SIPIJ

Signal & Image Processing : An International Journal is an Open Access peer-reviewed journal intended for researchers from academia and industry, who are active in the multidisciplinary field of signal & image processing. The scope of the... more

descriptionView Paper arrow_downwardDownload

Video Coding for Reliable Communications (PhD Thesis, 1999)

by Iain Richardson

This thesis presents the results of an investigation into the transmission of coded or compressed digital video information over communications networks. Practical networks and transmission media have limitations in the “Quality of... more

This thesis presents the results of an investigation into the transmission of coded or compressed digital video information over communications networks. Practical networks and transmission media have limitations in the “Quality of Service” that they can provide to transmitted data. The aim of this thesis is to examine the effect of Quality of Service limitations on the transmission of coded digital video and to investigate new techniques by which video quality can be improved in the presence of transmission impairments. The video coding algorithms are chosen from widely used International Standards for video coding and transmission.
The effects of Quality of Service limitations during transmission of coded video are investigated experimentally. Based on the results of these experiments, several different techniques to improve the quality of coded video are proposed and tested. These techniques include video rate control algorithms, scalable or layered coding as a means of prioritising selected components of coded video and modifications to the structure of coded video frames in order to minimise the propagation of errors.
Errors are introduced into coded video sequences and it is shown that their effects depend on the position of the error in the sequence. The error models include pseudorandom bit errors and cell errors as well as errors due to the User Network Interface in an Asynchronous Transfer Mode network. Prioritisation of selected components in coded video is investigated, by splitting coded video into several layers or by modifying the frequency of resynchronisation markers depending on the picture class of each video frame. The visual impact of transmission errors can be reduced in this way.
The effect of encoding parameters (including quantizer scale, slice resynchronisation interval and intraframe interval) on the bit rate of encoded video is quantified. A modified rate control algorithm that jointly controls quantization and intraframe interval is implemented and is shown to have the potential to improve the visual quality of constant bit rate coded video.
The results of these experimental investigations are discussed and evaluated in the context of a complete digital video communication system.

Figure 4 MPEG Group Of Pictures structure quence for applications that require random access.

More detailed cell loss models can be found in the literature, for example see papers in by the mean burst length.

Figure 13 Error spreading due to motion compensation

of each B-picture is about 2-3dB lower than that of its neighbouring I/P-pictures. Figure 15 PSNR of error-free decoded sequence

Figure 16 PSNR drop due to error in I-picture

Figure 17 PSNR drop due to error in 1* P-picture

in a GOP, and in Figure 19, where the cell loss occurs in the 3rd and final P-picture in a

drop due to cell loss in an I or P-picture. Figure 20 PSNR drop due to error in B-picture value are shown in Figure 20. The average is just over 3dB, significantly lower than the

Figure 21 PSNR of error-free and errored sequences

Figure 22 Drop in PSNR relative to error-free sequence

frames in the decoded sequence is 36.3 dB. Figure 23 Quality loss due to bit errors (Graduation)

Figure 24 PSNR of decoded MPEG video: bit errors (Nursery)

quality drops significantly. Figure 25 Quality loss due to cell losses (Graduation)

Figure 26 PSNR of decoded MPEG video: cell losses (Nursery)

(mainly P-pictures) and a smaller number between 250-400 cells (mainly I-pictures). Figure 28 Probability distribution

Figure 29 Contents of "leaky bucket": leak rate = 2 x MCR “burst” of cells, causing the bucket to fill up rapidly.

Figure 30 Contents of "leaky bucket": leak rate=1.2 x MCR which cells arrive has dropped.

of values from 160 (highest number of lost cells) to 220 (lowest number of lost cells).

Figure 32 Leaky bucket contents with adaptive rate control reduced variation in data rate compared to the unconstrained data rate shown in Figure 27.

Figure 34 Quantization step size Figure 34 shows how the quantization step size varies during the sequence.

Figure 35 Bit rate variation (JPEG encoding)

Figure 36 Bit rate variation of H.261 sequence

Figure 37 Bit rate for 1 GOP of MPEG sequence

Figure 38 Effect of encoding parameters on bit rate

Figure 41 Compexity measure for Graduation and Nursery

Figure 45 DC and 2 lowest AC coefficients a subset of the spatial frequencies. Figure 44 DC coefficients only

Figure 47 All coefficients Figure 46 DC and lowest 9 AC coefficients

pixels as shown in this example. Figure 51 Baseline JPEG with bit errors resynchronization marker. This means that an error can often cause a “streak” of corrupted

Figure 52 Mean PSNR of decoded sequences The mean PSNR for the two sequences (averaged across 5 experiments) is shown in Table

Sizes of 500-frame sequences (percentage of 20 20 20 sequence) Figure 53 Relative sizes of encoded sequences

Table 2 Visual quality grading scale Each observer is presented with a set of video sequences including reference sequences and

Table 3 Cell loss effects on decoded video (leak rate = 2 x mean cell rate)

In both sequences, the Standard Deviation is significantly reduced by the new method. The Table 5 Mean and Standard Deviation of PSNR

The Baseline JPEG sequence with 10° bit errors showed very poor visual quality with Table 7 Quality of decoded video sequences many obvious “streaks” across the decoded frames (see the example in Figure 51). JPEG

Each macroblock consists of 16 x 16 luminance samples and so each slice in the 20_20_20 Table 11 Test sequences

These slice sizes were chosen such that the file size (and hence the compression efficiency) Table 12 Varying slice sizes

The re-animated 20_20_20 sequence showed a number of obvious, large disturbances Table 13 Improvement in PSNR due to varying slice size

descriptionView Paper arrow_downwardDownload

Applications of MPEG-4: digital multimedia broadcasting

by Mara Grube

2001, IEEE Transactions on Consumer Electronics

The new MPEG-4 based DMB-system offers multimedia information with an excellent audio-and video quality to the mobile society. The combination of MPEG-4 encoding and decoding with the DAB-standard (Digital Audio Broadcasting) enables for... more

The DAB transmission system was developed for the broad- cast of audio and accompanying data by the European stan- dardisation activity Eureka 147 (ETS 300 401) [1]. It was designed for the demands of mobile receivers and offers an undisturbed reception even under difficult conditions. These results are due to the application of an OFDM-transmission system with a bandwidth of 1.5 MHz combined with a con- volution code for error protection. Fig. 1: The DAB/DMB transmission system The maximum data rate of the DAB system is 1.824 Mbit/s. It is therefore capable of multiplexing different services, which can include several audio channels and additional data. Examples are traffic news, text data, still images and moving pictures. These properties show the suitability of DAB in multimedia broadcasting. This will allow parallel transmission of audio, video, text and additional data as shown in Fig. 1. The elementary streams resulting from the MPEG-4 encoding stage are wrapped into an MPEG-2 Transport Stream (TS) [2]. Compared to the DAB-system [3], DMB improves the error protection with the additional

To gain a better understanding of the encoder structure, Fig. 2 gives a rough overview of the main modules within the MPEG-4 AAC Low Complexity Object Type:

Fig. 3: Nested loop concept for quantisation & coding The profiling process shows that the quantisation & coding module is the most time consuming part of the encoder. The main reason for this is the nested loop approach used by the original VM implementation. The goal of the concept, as displayed in Fig. 3 is the adaptation of the quantisation noise to the masking threshold considering the maximum number of bits available to encode the current frame. Since the codec works with a quasi-variable bitrate, it offers a slightly higher or lower bitrate than the average bitrate over a limited time period so as to encode critical sequences more accurately. Depending on the number of bits used in the foregoing frames, a “bit-reservoir control unit” deter- mines the “allowed” number of bits for the current frame. The first block initialises the quantisers of all frequency bands (subbands) with a low resolution, which results in a large quantisation error. Within the inner loop, the resoli- tion of each quantiser is increased (block 2) until the error energy of every subband is smaller than the error energy calculated in the psychoacoustic model (masking threshold).

Fig. 4: Improved quantisation & coding process This results in the inner iteration loop becoming obsolete. Also, the outer loop could be reduced to a maximum of five iterations without loss of audio quality, even if the s- quences are very critical. The basic ideas of the estimation algorithms can be found in the block diagram of Fig. 4.

Fig. 4 shows that the estimation process can be regarded as a kind of pre-processing (blocks 1 to 4) for the following outer iteration loop (blocks 5 to 9) [7]. The first block cal- culates the perceptual entropy from the error energy (mas k- ing threshold) and the signal energy. The second step esti- mates the bitrate from the perceptual entropy, which is needed to encode the frame optimally. The correlation be- tween the number of bits per frame, channel and the per- ceptual entropy was determined with help of simulations and is depicted in Fig. 5. Fig. 5: Bits per frame = f(PE)

To determine the quantiser resolution (scalefactors in terms of MPEG-4) effectively, an estimator was developed based on simulations by quantising and re-quantising a series of spectral values with different scalefactors. The results can be found in Fig. 6, where the quantiser error energy level is shown vs. the scalefactors used.

Fig. 8: Three-threaded working flow Fig. 8 shows the case of using three separate threads.

The decoder receives the bitstream in the size it has been encoded and decides when to resize the reduced frames. It is also possible to signalise those reduced frames on system level. But the method of managing the recognising and resizing of switched frames directly in the decoder has fi- nally been chosen [10]. The following figures show some results regarding the visual quality of a decoded test sequence encoded with ARM:

Fig. 11: Quantization parameter used by the encoder with and without ARM The gaps in the graph without ARM (Fig. 11) show that several frame skippings occurred during the encoding pro c- Fig. 10: Decoder with ARM

ess. The higher QP in this graph indicates a lower visual quality of the d ecoded sequence. Fig. 12: Buffer level of the encoding process with and without ARM

The “Stream Mode” scenario explained previously is as shown in Fig. 13.

Fig. 14: Encoder system To cope with the real-time processing demands, the operat- ing system on both encoder subsystems is SMP-Linux with a real-time extension.

Fig. 15: The decoder system With respect to mobile use, the important demands of the decoder are small size and an increased reliability of the bus-connectors. For these reasons, the decoder system uses Compact-PCI technology.

descriptionView Paper arrow_downwardDownload

Overview of the H.264/AVC video coding standard

by T. Wiegand and

2000, IEEE Transactions on Circuits and Systems for Video Technology

H.264/AVC is newest video coding standard of the ITU-T Video Coding Experts Group and the ISO/IEC Moving Picture Experts Group. The main goals of the H.264/AVC standardization effort have been enhanced compression performance and... more

Fig. 18. Evolution of H.264/AVC since August 1999 until March 2003. Top: QCIF sequence Foreman coded at 10 Hz. Bottom: CIF sequence tempete coded at 30 Hz. The legend in the top figure indicates the various versions that have been run with typical settings.

Fig. 1. Scope of video coding standardization.

Fig. 2. Structure of H.264/AVC video encoder.

Fig. 3. Parameter set use with reliable “out-of-band” parameter set exchange.

Fig. 5. Progressive and interlaced frames and fields.

Fig. 7. Subdivision of a QCIF frame into slices when utilizing FMO. Fig. 6. Subdivision of a picture into slices when not using FMO.

Top/Bottom Macroblocks in Field Mode Fig. 9. Conversion of a frame macroblock pair into a field macroblock pair. coding. Fora macroblock pair that is coded in frame mode, each macroblock contains frame lines. For a macroblock pair that is coded in field mode, the top macroblock contains top field lines and the bottom macroblock contains bottom field lines. Fig. 9 illustrates the MBAFF macroblock pair concept.

Fig. 10. (a) Intra_4x 4 prediction is conducted for samples a-p of a block using samples A-Q. (b) Eight “prediction directions” for Intra_4~ 4 prediction. Fig. 11. Five of the nine Intra_4~ 4 prediction modes.

Fig. 12. Segmentations of the macroblock for motion compensation. Top segmentation of macroblocks, bottom: segmentation of 8x 8 partitions.

Fig. 13. Filtering for fractional-sample accurate motion compensation. Upper-case letters indicate samples on the full-sample grid, while lower case samples indicate samples in between at fractional-sample positions.

Fig. 14. Multiframe motion compensation. In addition to the motion vector, also picture reference parameters (A) are transmitted. The concept is also extended to B slices.

Fig. 16. Principle of deblocking filter.

Fig. 17. Performance of the deblocking filter for highly compressed pictures (a) without deblocking filter and (b) with deblocking filter.

descriptionView Paper arrow_downwardDownload

A new modified fast fractal image compression algorithm

by Mahdi Salarian

In this paper, a new fractal image compression algorithm is proposed, in which the time of the encoding process is considerably reduced. The algorithm exploits a domain pool reduction approach, along with the use of innovative predefined... more

degrading some of the others. To evaluate compres- sion algorithms and also compare them pairwise, some measures were defined as briefly explained below.

The last row is cited from Ref. 16, and the two others are from our experiments. As shown above, the entropy-based method is superior to the Saupe’s method, so we can compare our algorithm to the entropy-based one. To understand the significance of the size of the domain pool, one can consider a 512 x 512 image, with domain block of size 16 x 16 with overlap of 4 pixels; in this case, there are 15 625 domain blocks that must be checked if they match to each range block. A huge number of computations in this case are obvious. By restricting the domain pool to a number of less than 500 members, for example, the large amount of computations given above will be decreased by about 30 times.

number of these permutations, is given in equation (13). 3.2 The effect of contrast scaling factor S

values. Obviously, restricting the size of the search set of S to a two-member set will decrease the encoding time considerably. To find a realistic estimate for S, a large number of experiments with exhaustive search for S$ were performed. One can easily see that at Step 1, the optimal S$ has often lower value than 0.1, independently on the image, so for this step, we let S=0.1. At Step 2, the optimal value of S is less than 0.5, so here we can choose S from {0.2, 0.4}. At Step 3, S has approximately uniform distribution across [0, 1], so in order to determine distinct values in this phase, we chose S from {0.3, 0.8}. For Step 4, S gets higher values in [0, 1], so we can choose S from {0.5, 0.9}. In this step, the blocks’ sizes are 2 x 2, and therefore, they can be encoded very accurately. drastically decreased as the following quantitative comparison demonstrates

rp 5 Histogram of S with pool size as parameter for (a) step 1 and (b) step 2

approaches. Figure 7 shows the associated data of a range block stored as compressed data. The coordination of domain blocks is kept in a list, and the entries of the list are used as the address. It is clear t hat the addresses of the range blocks need not to be stored, since they follow the same regular order in the image and in the stored data. They are stored in order from left to right, and rows from up to down. The last field is S with 1-bit length for Steps 2-4, indica stored ing one of the two predefined values for S. The data frames do not contain this field if the step is 1, since in this case there is only one default value for S.

To judge the performance of this algorithm by the human eye, the results of applying different techni- ques on three of the used images are shown in Fig. 10.

Table 1 Comparison between Saupe’s method and entropy-based algorithm (DN denotes the size of the domain pool) As one can easily see, when the pool size has small values, PSNR, SSIM and compression ratio are very small, which are not desirable, but they grow rapidly when the pool size is increased. For the greater values of the pool size, the growth of the two measures becomes slower, and increasing the pool size only increases the encoding time. These results verify the validity of the domain pool reduction approach. Using these three figures, one can approximately adjust the performance attributes of the algorithm by adjusting the pool size.

Table 5 The comparison results for Baboon

descriptionView Paper arrow_downwardDownload

Cost-Efficient Virtual Machine Management: Provisioning, Admission Control, and Consolidation

by Adnan Ashraf

2014

One of the main challenges in Software Engineering is to cope with the transition from an industry based on software as a product to software as a service. The field of Software Engineering should provide the necessary methods and tools... more

One of the main challenges in Software Engineering is to cope with the transition from an industry based on software as a product to software as a service. The field of Software Engineering should provide the necessary methods and tools to develop and deploy new cost-efficient and scalable digital services. In this thesis, we focus on deployment platforms to ensure cost-efficient scalability of multi-tier web applications and on-demand video transcoding service for different types of load conditions.

Infrastructure as a Service (IaaS) clouds provide Virtual Machines (VMs) under the pay-per-use business model. Dynamically provisioning VMs on demand allows service providers to cope with fluctuations on the number of service users. However, VM provisioning must be done carefully, because over-provisioning results in an increased operational cost, while under-provisioning leads to a subpar service. Therefore, our main focus in this thesis is on cost-efficient VM provisioning for multi-tier web applications and on-demand video transcoding. Moreover, to prevent provisioned VMs from becoming overloaded, we augment VM provisioning with an admission control mechanism. Similarly, to ensure efficient use of provisioned VMs, web applications on the under-utilized VMs are consolidated periodically. Thus, the main problem that we address is cost-efficient VM provisioning augmented with server consolidation and admission control on the provisioned VMs. We seek solutions for two types of applications: multi-tier web applications that follow the request-response paradigm and on-demand video transcoding that is based on video streams with soft realtime constraints.

Our first contribution is a cost-efficient VM provisioning approach for multi-tier web applications. The proposed approach comprises two sub-approaches: a reactive VM provisioning approach called ARVUE and a hybrid reactive-proactive VM provisioning approach called Cost-efficient Resource Allocation for Multiple web applications with Proactive scaling. Our second contribution is a prediction-based VM provisioning approach for on-demand video transcoding in the cloud. Moreover, to prevent virtualized servers from becoming overloaded, the proposed VM provisioning approaches are augmented with admission control approaches. Therefore, our third contribution is a session-based admission control approach for multi-tier web applications called adaptive Admission Control for Virtualized Application Servers. Similarly, the fourth contribution in this thesis is a stream-based admission control and scheduling approach for on-demand video transcoding called Stream-Based Admission Control and Scheduling. Our fifth contribution is a computation and storage trade-o strategy for cost-efficient video transcoding in cloud computing. Finally, the sixth and the last contribution is a web application consolidation approach, which uses Ant Colony System to minimize the under-utilization of the virtualized application servers.

NB: The permanent address of the publication (PDF file) is http://urn.fi/URN:NBN:fi-fe2014092244819

Figure 2.1: Service and deployment models in cloud computing

Figure 3.1: System architecture of the proposed VM provisioning, admission control, and consolidation approaches for multi-tier web applications

Figure 3.2: System architecture of the proposed VM provisioning and ad- mission control approaches for video transcoding

Figure 3.3: The estimated equilibrium point between the storage cost anc the transcoding cost of a transcoded video

Figure 3.4: A simple example to motivate the need to consolidate multiple web applications in a cloud-based shared hosting environment

Figure 4.1: Relationship among original publications

Figure 1: System architecture of the cloud-based on-demand video transcoding service The overflow occurs if the video transcoding rate exceeds the video play rate and the capacity of the buffer. Likewise, the buffer underflow may occur when the play rate exceeds the transcoding rate, while the buffer does not contain enough frames either to avoid the underflow situation. Since the main focus of this paper is on video transcoding, we assume that the streaming server is not a bottleneck. ey ag - i. oe

Figure 2: The estimated equilibrium point between the storage cost and th transcoding cost of a transcoded video is transcoded once and stored in the video repository, then initially the compu- tation cost is higher than the storage cost. However, with the passage of time, the storage cost continues to increase until it becomes equal to the computation cost and then it grows even further unless the video is removed from the video repository. Thus, if the video is deleted before its estimated equilibrium point and then it is subsequently requested, the computation cost will increase due to unnecessary re-transcoding. Likewise, if the video is stored beyond its estimated equilibrium point and then it does not receive a subsequent request, the storage cost will increase unnecessarily.

Table 1: Summary of concepts and their notation for VM allocation algorithms

Finally, the total cost and popularity score S,, of a video 7; is calculated by accumulating the new cost and popularity score NS;,, of the said video over time. That is, for each new request of a transcoded video 7;, we obtain the previous value of the total cost and popularity score S,, of the transcoded video, calculate NS,,, and then add them together to produce the new value of the S,,. Moreover, the total cost and popularity score of a video that was not stored previously is set to NS,,. The total cost and popularity score S,, determines the exact duration for which a video 7; should be stored. The pseudocode for score calculation is presented in Algorithm 3.

If the inequality holds, it implies that it is cost-efficient to delete the transcoded video. Therefore, the video is removed from the video repository. However, if the inequality does not hold, it indicates that it is not cost-efficient to delete the video. Therefore, the video is not removed. Moreover, the cost and popularity score S,, is decremented in accordance with the length of the time interval to reflect the passage of time. In this way, when a popular video loses its popularity, it starts losing its cost and popularity score as well until it is removed from the video repository or it gets some new requests to regain its popularity. The pseudocode to decrement cost and popularity score S,, and to remove a video is given as Algorithm 4.

Table 4: Summary of results The average number of transcoding servers was 94, the total transcoding cost was $4179.12, the total storage cost was $4090.56, and the total storage size was 34.19 terabytes. Since the usage based strategy stores only popular videos, the storage cost of the usage based strategy was slightly less than that of the store all strategy. Therefore, the results indicate that the usage based strategy is cost-efficient when compared to the store all strategy. However, since it does not account for the computation and the storage costs, it may remove some videos that have a high transcoding cost.

descriptionView Paper arrow_downwardDownload

Video Coding for Streaming Media Delivery on the Internet

by karina dewanti

2001, IEEE Transactions on Circuits and Systems

We provide an overview of an architecture of today's Internet streaming media delivery networks and describe various problems that such systems pose with regard to video coding. We demonstrate that based on the distribution model (live or... more

Fig. 1. Communication between RealAudio server and RealA udio player.

Fig. 2. Streaming protocols used by RealSystem G2, and its interoperability with other standards-based systems

are one-way transmission and nonguaranteed delivery of information. In addition, the server does not typically know how many clients are subscribed to the broadcast and/or their actual connection statistics. A possible way to serve clients with different connection speeds is to simulcast several independent encoded versions (streams) of the source targeted for different bit rates, and let clients decide which stream to use. ting can be initiated by a client connecting to a local splitter (or network cache acting as a splitter) which, if not active, trans- fers request to an upper tier splitter, and so on, until it reaches the nearest active splitter. Once such a splitter is found, it can start transmission of the requested information down through the chain of intermediate connections to the client. We call this model pull splitting. In the case where a splitter is used as part of a multiple-access delivery network, it can establish connections to several geographically distributed upper-tier splitters. We call such a delivery process multiple-access splitting.

Fig. 3. Delivery of live and/or simulated live content.

Fig. 5. Motion-compensated hybrid coder.

Fig. 6. Adaptive Stream Management in RealSystem 8.

Fig. 9. RD comparison charts for clip “Foreman,” QCIF, 7.5 fps.

Fig. 10. RD comparison charts for clip “News,” QCIF, 15 fps.

Fig. 11. Client-side video processing in RealSystem 8.

COMMUNICATION PROCESSES IN STREAMING MEDIA An even more interesting coding problem arises with mul- tiple-access splitting. Such splitters request and receive infor- mation from multiple sources, which can be considered a form of multiple-access channel [24]. In turn, the information re- ceived by splitters is intended for multiple clients. The combi- nation of both requirements leads to a variant of the problem of source coding for multiterminal networks [26], [24].

descriptionView Paper arrow_downwardDownload

An Investigation of Dehazing Effects on Image and Video Coding

by Dũng Võ

2000, IEEE Transactions on Image Processing

This paper makes an investigation of the dehazing effects on image and video coding for surveillance systems. The goal is to achieve good dehazed images and videos at the receiver while sustaining low bitrates (using compression) in the... more

Fig. 1. Dichromatic model. [3], the authors take a unique approach based on the human-vi- sual-system model by loosely using a Retinex method [18], [19] while operating in the discrete cosine transform (DCT) domain. Most of the transform-domain techniques may be applied either at the encoding end, i.e., Pre, or the decoding end, i.e., Post, but the performance comparisons are not explored.

As shown in Fig. 2(a), the closer pixels (less atmospheric veiling [14]) are darker than hazier background pixels that are farther away. Notice that, comparing (15) with (16), even if there was one foreground pixel at the extreme edge of neighborhood Q(m, n), the DCP will choose the foreground pixel value & ¢ at (m,n), whereas the MDCP method will choose the background pixel Zy. If no refinement is used, then this will cause halo arti- facts at occlusion boundaries for the case of using the DCP. In

Fig. 2. Examples of occlusion boundaries. (a) Original image. (b) Dehazed using unrefined DCP method (9). (c) Dehazed using our proposed MDCP method (12). (d) Dehazed using method from [14]. Notice in (b) and (d) that there are halo artifacts around the leaf edges. The halo artifacts are not as dominant in (c). (a) Hazy image x. (b) xp. (c) xy. (d) xr.

The next step is to apply an N x N forward type-2 DCT. In the direct form, the (7,7)th N x N block of the transformed luminance is In [12], the dichromatic model is used to explain the contrast at isodepths (smooth depth regions) due to haze or fog being reduced. If we measure the gradient of the isodepth, threshold the result, and sum them, i.e., ||V - ||, we get a contrast measure. Therefore, we have the following relationship: Once the luminance is obtained, the values are offset to be within the range of [-2",2” — 1] for (P + 1) bit data. For clarity purposes, we will not make a notational difference and, from this point on, assume that the luminance values for 7% and y are subtracted by 2”. This brings us to the junction between the “Level and Offset” and “ N x N Block DCT” blocks in Fig. 3.

Fig. 4. Comparison of the dehazed image using Pre and Post methods with similar file sizes. (a) Synthesized Haze. (b) Dehazed (no compression). (Red box) Region of interest used for visual comparison. (c) Uncompressed dehazed image. (d) Pre method. (e) Post method. image for the Post method. A cropped region of the results is dis- played in Fig. 4. Although we set the quality low enough such that the Pre method resulted in a smaller image size, the Post method suffered more from ringing artifacts as evident around the building edges in Fig. 4(e). We can also see more blocking artifacts in the sky region above the buildings (see Fig. 5).

Fig. 7. Performance comparison of the deblocked Pre and Post methods using JPEG and WILD. Fig. 6. Measure of blocking artifacts for each method at different image sizes.

Fig. 5. Real-world example of Pre and Post using MDCP (12) and JPEG compression. (a) Original image. (b) Post method with MDCP. (c) Pre method with MDCP. (d) Detected block artifacts in Post image using [26]. (e) Detected block artifacts in Pre image using [26].

Fig. 9. Luminance PSNR versus bitrate using H.264.

Fig. 8. (a) Frame from the original video sequence. (b) Snapshot from the de- hazed video. (Red box) Region used for closer comparisons. [(c)—(e)] Compar- isons of the Pre and Post methods with H.264 compression at 250- and 500-kb/s bitrates. (a) Original video frame. (b) Dehazed video frame. (Red box) Region of interest used for comparisons. (c) 250-kb/s Post. (d) 250-kb/s Pre. (e) 500-kb/s Post. (f) 500-kb/s Pre.

Fig. 10. Measure of blocking artifacts for each method and bitrate. The metric was applied to every frame in a video sequence and then averaged.

Fig. 11. Plot of ITU-R-averaged marks with error bars versus rank. bits. The preserved details using Pre method contributed to the noticeable improvement in visual quality. We performed a subjective test with 20 different observers. These observers were presented with a test that displayed two videos side by side with all combinations such that the pool of randomly selected videos were different with respect to their bitrates and method (Pre or Post). After each playback, the ob- server was asked to compare the left video to the right. The choices the observers were allowed to select for comparing the left video sequence to the right were adjectival categorical judg- ments ranging from Much worse, Worse, Slightly worse, The same, Slightly better, Better, and Much better. This type of test is called the stimulus comparison adjectival categorical judgment (SCACJ) [36]. After processing the data, only the results from one observer was rejected due to rejection criteria in [36]. The remaining 19 observer results were used to compute the ITU-R statistics. For the pool of video sequences, we choose a range of bitrates that were close enough together to give us a good res- olution in our results but limited the maximum bitrate to 1000 kb/s because it was difficult beyond that to distinguish from the uncompressed sequence. — ArmlanmntT ay +1 414. rmmo4t14 #é6oTT ore es azed Video ITU-R Averaged Mark ITU-R Left Confidence Border ITU-R Right Confidence Border ITU-R Std. Dev

“OMPARISON OF AVERAGE MV. AND RESIDUAL BITS PER FRAME Table I. The bit ratio for each method, i.e., Post/Pre, is added to the table to make it easy to see the bit allocation relationships.

over the Post method for any bitrate. The error-bar plot (using the rankings from Table I) of the ITU-R results in Fig. 11 shows two significant overlaps between the uncompressed (ranked first) and 1000-kb/s Pre sequences (ranked second). and the 250-kb/s Pre (ranked fifth) and 1000-kb/s Post (ranked sixth) sequences. The observers had difficulty distinguishing between the uncompressed enhanced sequence and the pre-en- hanced and compressed sequence. In addition, the observers had difficulty distinguishing between the 250-kb/s Pre and 1000-kb/s Post sequences. This is a significant compression difference by a factor of 4. If one is only concerned with a dehazed video sequence and can decide whether to enhance before or after compression, our results show choosing the Pre method can operate just as well as the traditionally chosen Posi method while operating in significantly lower bitrates.

descriptionView Paper arrow_downwardDownload

Robust video transmission using H. 264 and real-valued BCH frames

by Pierre Duhamel

2004

With the increasing use of multimedia technologies, video coding requires higher performances as well as new features. To address this need, the latest ITU-T video coding standard, H.264, has been developed. The motion compensation... more

descriptionView Paper arrow_downwardDownload

Motion compensation for very low bit-rate video

by Sérgio Faria

1995, Signal processing: image …

A combination of spatial transformation and image segmentation is used to compensate for non-uniform intensity changes in moving scenes. The method efficiently tracks movements such that the motion vectors alone can be employed to... more

descriptionView Paper arrow_downwardDownload

An Efficient Intra Mode Selection Algorithm For H.264 Based On Fast Edge Classification

by zhenyu WEI

2007

The H.264/AVC is the newest video coding standard recommended by ITU-T and MPEG. Compared with all existing video coding standards, H.264 can achieve superior performance by using many advanced techniques. Intra mode selection is an... more

Fig. 2. RDO computation. re SESH SS aia be eer ee “Let C denote a 2’ x 2 block. Bo, B;, Bz and B3 denote intensity sum of four non-overlap sub-block in a given block C, which are shown in Fig. 4. Then, the four NHT From (1) and (2), it can be known that choosing I16MB means that fewer bits are used to signal the mode type and other information, but the residue may contain higher energy, especially in sequences with more details. On the other hand, choosing I4MB may give a lower energy residual after intra-prediction but it needs to use more bits to signal the mode types and other information.

Fig. 4. Possible edge orientation in the given block.

Fig. 3. (a) The multi-resolution representation of 2-D NHT. (b) Block features at different levels.

Fig. 5. The framework of the edge classification method.

where n stands for the number of encoded macroblocks in past, N, Z;, E, are the values of the three corresponding components of the kth MB, R; is the real consumed bits for coding the quantized coefficients of the kth MB.

Fig. 6. Correlation between actual bits and predicted bits. For intra-mode decision, H.264/AVC combines luma and chroma components together to perform RDO opera- tion since a symbol named CBP is related to not only chroma intra-mode type, but also luma intra-mode type. In order to obtain the accurate RDcost where bits for

Fig. 7. The total number of RDO operations in inter mode decision [2]. Since the bits for encoding CBP is very few, luma and chroma components can be considered approxima- tively independent when performing intra-mode selec- tion. By trading off accuracy vs. complexity in intra-mode decision, RDO computation for chroma and luma compo- nents can be performed separately to choose the best chroma mode and luma mode. In this case the total number of computed RDcost is given by (13), and only 164 RDO operations are needed instead of original 640

Fig. 8. The total number of RDO operations in intra-mode decision [2]. RDO operations.

Fig. 9. Rate-distortion curves (for all I-frames): (a) Foreman (QCIF). (b) Carphone (QCIF). (c) Bus (CIF). (d) News (CIF).

Fig. 10. Rate-distortion curves (IPPP... Mode): (a) Foreman (QCIF). (b) Carphone (QCIF). (c) Bus (CIF). (d) News (CIF).

The relationship between the prediction modes and the edge modes for 4 x 4 luma block Table 1 type will be skipped and computational cost will be saved. The procedure for checking the homogeneity of the macroblock is described as follows. First the three AC NHT coefficients are computed for the given macroblock. If any one of the following three conditions is satisfied, this macroblock will be considered as homogeneous. mode 0, mode 3 and mode 7 are chosen as the candidates according to Table 1.

Coding performance compared with JM10.1 RDOp, (all intra-frame) Table 3

Coding performance compared with JM10.1 RDOo,(All intra-frame) Table 4

Coding performance compared with JM10.1 RDOo, (IPPPP....mode) Table 5

Coding performance compared with JM10.1 RDOo, (IPPPP....mode)

descriptionView Paper arrow_downwardDownload

An Eigenspace Update Algorithm for Image Analysis

by Shivkumar Chandrasekaran and

1997, Graphical Models and Image Processing

matrix factorizations appeared in 1974 [3]. However, until recently there was no fast and stable updating algorithm During the past few years several interesting applications of eigenspace representation of images have been proposed.... more

descriptionView Paper arrow_downwardDownload

Evaluating the CU-tree algorithm in an HEVC encoder

by Vladimir Grozman

CU-tree (Coding Unit tree) is an algorithm for adaptive QP (quantization parameter). It runs in the lookahead and decreases the QP of blocks that are heavily referenced by future blocks, taking into account the quality of the prediction... more

descriptionView Paper arrow_downwardDownload

Video Object Detection and Tracking using kalman filter and color histogram-based Matching algorithm

by IJERD Editor

Video coding is most popular from 1990 starting with video teleconferencing, videophone, videoCD, DTV, HDTV,Multimedia framework,etc.,Video compression becomes most important when considering the space requirement, channel width and the... more

Fig. 1.1: Inter-object occlusion and separation in the moving objects An overview of general tracking. system is shown Fig.1.1.This illustration depicts the behavior of th moving objects in real-time where the confusing situations are observed in different intervals, such as inter object occlusion or separation.

Use the Measurement matrix source parameter to specify how to enter the measurement matrix values. If yo select Specify via dialog, the Measurement matrix parameter appears in the dialog box. If you select Input por <H>, the H port appears on the block. Use this port to specify your measurement matrix[4,5-6]. Fig 2.8(b shows how well the Kalman Filter was able to estimate the position, in spite of the large measurement noise.

Fig.4.1:; Workflow of our project is showing the implementation In this approach, each tracker and moving object pair is associated with manager-oriented logical framework which verifies the status of each moving object during tracking using conventional feature-based technique (i.e. area, boundary and center of gravity) and with the proposed CWHI-based technique. Each mov- ing object can have the following _ status. For example, isNew, isReachAtEnd, is Occluded/ is Occluder, and is Reappear with two integrity constraints (i.e. is Overlap and is Split). The “manager” as a logical controller man- ages the entire tracking process and assists trackers (i.e., Kalman filter). More precisely, it assists Kalman track- er during tracking even if nonlinear conditions (i.e., inter-object occlusion and separation) occurred. Fig.4.1 illu- strates the workflow of the logical-framework of the proposed research. The “manager” is responsible for han- dling the two cases (i.e. inter-object occlusion and separation) with a “normal case” when no confusion is ob- served. These are: In the most ideal situation, there is no inter- object occlusion and separation. This is the simplest and ideal scenario in tracking, where a newly detected moving object “isNew’ is assigned a new tracker. When the moving object is reached at “isReachAtEnd”, then both the moving object and the tracker is removed.

V. EXPERIMENTAL RESULTS tive moving objects in the sequence by calculating the “measure of similarity”. Based on the “measure of simi- larity”, a tracker is assigned to its respective moving object under confusions. The result proves that tracking of multiple objects is possible even if non-feasible situation appeared using the proposed intelligent feature-based approach.

Video Object Detection and Tracking using kalman filter and color histogram-based... RESULTS OF MOVING CARS AND GRAPH

The presented work is based on our initial research analysis on multi-object tracking using stochastic tracking and feature-based matching techniques. These results motivate us to further investigate histogram- based matching techniques together with stochastic tracking algorithms. In particular, more distinctive tech- niques will be exploited such as fuzzy logic or AdaBoost classifier with histogram to handle more complex real- time scenarios. However, the proposed algorithm performs very well on tested videos achieving robust tracking with 97.3% accuracy and 0.07% covariance error. Fig.5.2(h): Results of moving cars with motion trajectories

descriptionView Paper arrow_downwardDownload

Subjective quality of SVC-coded videos with different error-patterns concealed using spatial scalability

by Yohann Pitrey

2011, … (EUVIP), 2011 3rd …

Degradation of network performance during video transmission may lead to disturbing visual artifacts. Some packets might be lost, corrupted or delayed, making it impossible to properly decode the video data on time at the receiver. The... more

Fig. 1. Impairment patterns with associated Mean Opinion Scores and QP values. For each HRC, the two dotted lines denote the base and enhancement scalable layers. The solid line represents the layer from which the current frame is displayed (base layer means impairment). We vary the number of impairments, the total length of the impairments, the interval between impairments and the quality of the base layer (QP values appear in the two columns on the right.

Fig. 2. Comparison of the Mean Opinion Scores for the four evaluated factors in our experiment.

descriptionView Paper arrow_downwardDownload

Subsequence video coding for improved temporal scalability

by Moncef Gabbouj

2005

Compression efficiency and bitrate scalability are among the key factors in video coding. The paper introduces novel sub-sequence coding techniques for temporal scalability. The presented coding schemes provide a wider range for bitrate... more

Figure 1. Example of sub-sequences: coding pattern “IpPpP” (The numbers in the figure indicates the output order and the number of reference frames is 3.)

Figure 2. Rate-distortion curves for Pairs (CIF @ 30 Hz)

TABLE II. AVERAGE RATE-DISTORTION DIFFERENCES (DB) AT FULL FRAME RATE (a: IpPpP vs. IPPP, b: IpPpP vs. IppP, c: IbBbP vs. IbbP, d: IppP vs. IbbP, e: IpPpP vs. IbBbP. A positive value implies the former scheme outperforms the latter)

descriptionView Paper arrow_downwardDownload

Very low bit-rate video coding

by Manoranjan Paul

2004, 2004 IEEE Region 10 Conference TENCON 2004.

I would like to express my sincere gratitude and profound indebtedness to my main supervisor Manzur Murshed and associate supervisor Guojun Lu for their constant guidance, insightful advice, helpful criticism, valuable suggestions,... more

Figure 1.1: An example on how pattern-based coding can exploit the intra-block temporal redundancy in improving coding efficiency. already VLBR video coding option.

Figure 1.2: An example of pattern approximation for the Miss America standard video sequence, (a) frame number one, (b) frame number two, (c) detected moving regions, and (d) results of pattern approximation.

Figure 2.1: Generic block-based video encoder structure.

Figure 2.2: Generic block-based video decoder structure. DCT (IDCT) are used to form the reference frames which are stored in the frame memory for

Figure 2.3: A typical GOP for digital video coding.

Search window of a reference frame distance-dependent thresholding search proposed by Sorwar, Murshed, and Dooley [124]. Figure 2.4: Block motion vector estimation.

Figure 2.10: Relative frame size of CIF (largest), QCIF, and Sub-QCIF (smallest) format using the first frame of the Miss America video sequence as the example.

Figure 2.11: Variable block sizes for the H.264 standard: (a) 16x16, 16x8, 8x16, and 8x8, (b) 8x8, 8x4, 4x8, and 4x4.

Figure 2.12: Graphical representation of 4x4 grey prediction modes [109]. 8, the predicted samples, are obtained from a weighted average of the prediction samples A-M. arrows in Figure 2.12 indicate the direction of prediction in each mode. For example, modes 3- The nine modes are as follows: i) mode 0 (vertical): where the upper samples A, B, C, and The nine modes are as follows: i) mode 0 (vertical): where the upper samples A, B, C, and D are extrapolated vertically; ii) mode 1 (horizontal): where the left samples L, J, K, and L are extrapolated; iii) mode 2 (DC): where all samples in the predicted block are obtained by the mean of samples A...D and I...L; iv) mode 3 (diagonal down-left): where the samples are interpolated at a 45° angle between lower-left and upper-right; v) mode 4 (diagonal down- right): where the samples are extrapolated at a 45° angle down and to the right; vi) mode 5 (vertical-right): where there is extrapolation at an angle of approximately 26.6" to the left of the vertical (width/height = 1/2); vii) mode 6 (horizontal-down): where there is extrapolation at an angle of approximately 26.6° below the horizontal; viii) mode 7 (vertical-left): where there is extrapolation (or interpolation) at an angle of approximately 26.6° to the right of the vertical; ix) mode 8 (horizontal-up): where there is interpolation at an angle of approximately 26.6° above the horizontal.

Figure 2.13: Vector Quantisation coder and decoder. The set Y is called the codebook and each element of Y is called a codeword. Figure 2.13 shows the functional block diagram of a basic VQ video coder (blocks shown

Figure 2.14: Basic schematic diagram of model-based image coding. In image modelling [3][125][17][11][142], a geometric wireframe model is used for the

Figure 2.18: Block diagram of pattern-based video coding encoder.

Figure 2.19: Block diagram of pattern-based video coding decoder. Very Low Bit Rate Video Coding: A Review

Figure 2.20: Rearrangement of the residual errors so that only one 8x8 block is needed to accommodate the 64 pixels position in a pattern [141]. patterns do not need rearranging because they are already in one 8x8. Since DCT is applied on

Figure 2.21: Standard QCIF video sequences with their motion type, (a) Miss America (“talking heads” motion), (b) Suzie (hand and head movements), (c) MotherDaughter (two heads movements), (d) Carphone (fast object motion), (e) Foreman (object translation and panning), (f) Salesman (head- shoulder-hand movements), and (g) Claire (“talking head” motion). example, Carphone possesses relatively high background object motion compared with either

Figure 3.1: Percentage of different size blocks (a) and sub-blocks (b) selected by H.264 at different bit rates for the Salesman sequence.

Figure 3.2: (a) Frame number two of the Miss America sequence, (b) different size blocks, and (c) and (d) different size blocks of frame number two at low and high bit rate respectively.

Figure 3.3: Example MB (a) with quadrants, (b) with few moving pixels in I and II quadrants, and (c) with huge moving pixels in I, II, and IV quadrants. 3.3(c) is unnecessarily matched with the pattern codebook as there is no ‘1’ in quadrant II.

Figure 3.4: Increase in the number of RMBs (%) by the Fixed-8 algorithm with parametric MB classification compared to the quadrant based classification.

Figure 3.5: Number of RMBs generated by the Fixed-8 algorithm applied on the first 100 frames of the Miss America video sequence for different 5 values in parametric MB classification.

parametric MB classification, unless stated otherwise. Figure 3.6: Percentage of increased number of RMBs vs. pattern codebook sizes for seven standard video sequences.

Figure 3.7: The extended pattern codebook of 32 regular shaped, 64-pixel patterns, defined in 16x16 blocks, where the white region represents ‘1’ (motion) and the black region represents ‘0’ (no motion). shown in Figure 3.7, where the first eight represents the patterns used by Wong et al. regularity feature is added to limit the pattern codebook size. The full 32-pattern codebook is

Figure 3.8: Percentage of increased RMBs generated by the Fixed-32 algorithms compared to the Fixed- 8 algorithm using parametric MB classification with O= 120.

Figure 3.9: Change in the frequency of RMB captured by each pattern while the second best similar pattern is also considered along with the best similar pattern and the one leading to higher overall compression is selected.

Figure 3.10: Coding efficiency curve for the Miss America video sequence using the Fixed-8, Fixed-32, Fixed-32 with VLC (Fixed-32_VLC), Fixed-32 with VLC and new parametric MB classification (Fixed-32_VLC_PMB), and the Fixed-32 with EVLC and new parametric MB classification (Fixed- 32_EVLC_PMB). fraction of bit per RMB.

Figure 3.11: (a) Similarity example for a moving region M of a CRMB and pattern P 1; (b) two non- overlapping areas (black) relevant to the PISM; (c) non-overlapping area (black) relevant to the PESM. from a set of CRMBs: heuristic is therefore justified in order to classify the approximately similar number of RMBs

Figure 3.12: Flowchart of an efficient computational strategy on (a) the PISM, (b) the PESM, where NOP means no operation. regard to whether an MB is classified as an RMB or AMB.

Figure 3.13: Coding efficiency curves for the standard video sequences using the Fixed-32 (with new parametric MB classification, efficient VLC, and PESM), Fixed-8, and the H.264*. Enhanced Pattern Based Video Coding

Figure 3.14: Coding efficiency curves for the standard video sequences using the Fixed-32 (with efficient VLC, parametric MB classification, and PESM), Fixed-8, and the H.264*.

Figure 3.15: Frame level PSNR in the Miss America sequence using the H.264*, Fixed-8, and the Fixed-32 algorithms for target bit-rate 22 Kbps. Enhanced Pattern Based Video Coding

Figure 3.16: (a) Miss America frame number three, (b)—(d) reconstructed frames using the H.264*, Fixed-8, and the Fixed-32 algorithms respectively, (e)-(g) frame differences (x3) of (b), (c), and (d) respectively with respect to (a). Enhanced Pattern Based Video Coding

Figure 3.17: (a) Claire frame number three, (b)—(d) reconstructed frames using the H.264*, Fixed-8, and the Fixed-32 algorithms respectively, (e)-(g) frame differences (<3) of (b), (c), and (d) respectively with respect to (a). Enhanced Pattern Based Video Coding

controlling the number of patterns for each CRMB.

Figure 4.2: Speed-up factor, 17min ) for different minimum number of patterns, 77min on the Miss America sequence.

Figure 4.5: Speed-up factor (11) and (32) on seven standard sequences. Lemma 4.5: Fixed-32 algorithm requires 256+(M +33)A “add-equivalent” operations. be proven by means of a similar argument to that used in [141].

Figure 4.7: Percentage of RMBs where the same pattern is selected by the RTPS(11) & Fixed-32, RTPS(4) & Fixed-32, and the Fixed-8 & Fixed-32 algorithms.

Figure 4.8: Percentage of RMBs in the Miss America sequence where the best similar pattern (captured by the Fixed-32 algorithm) is selected by the RTPS(7min) algorithm using the proposed iso-7Tz technique and the straightforward technique of selecting of the first 77a,2(77min) best relevant patterns. sorting for lower values of 77min. straightforward selection of the first 7ave(77min) best relevant patterns in the CPC through

Figure 5.1: 64 patterns generated using definition where white indicates the moving region and black indicates the static region. while edges of the patterns are aligned to the block boundaries as much as possible. in Figure 5.1 are considered so that the inter-relations among them are kept to a minimum

Figure 5.2: Patterns generated using (a) Euclidean distance, (b) Manhattan distance, and (c) Chebyshev distance.

Figure 5.3: RMB frequencies of the best eight patterns among 64 patterns shown in Figure 5.1. this sense, definition based pattern generation would, ultimately, prove a good solution.

Figure 5.6: Actual and approximated (dotted lines) values of iterations of EVPS(/ € {4,8,16,32}, ye {51%,...,100%}) algorithm for the Suzie sequence. Efficient Pattern Codebook Downsizing

Figure 5.7: The intermediate pattern set size of the EVPS(4, 75%) algorithm for the Suzie sequence where doted line is assumed as intermediate pattern set size for calculation. VLC of pattern numbers. Considering a-4A selection iterations and Q,| =a-i+l1, the VPS

Figure 5.7 shows the example of 1 = 4 as it represents the worst case of pattern elimination.

Figure 6.2: (a) 3-D representation of pixel frequency of one of the eight clusters of CRMBs obtained for Miss America sequence; (b) its corresponding 2-D top view projection; and (c) the generated pattern for this cluster by the CPG algorithm. represents only local optima. pattern is constrained to represent the entire cluster. However, such optimality, in reality, pattern generated by the CPG algorithm guarantees an optimal similarity measure if only on

Figure 6.3: 64-pixel size patterns extracted from video sequences by the CPG algorithm. patterns. one of the main reasons for the superior coding efficiency achieved by the arbitrary shaped

Figure 6.4: Percentages of RMBs in a cluster using the pattern generated by the same cluster in APVC algorithm for different number of processing frames where PC size is eight.

Figure 6.5: Performance of APVC algorithm with different PC sizes where each pattern has 64-pixel. omputational complexity in calculating the similarity metric calculation due to the larg

Figure 6.6: Performance of the APVC algorithm using different similarity thresholds for the Miss America sequence with eight patterns.

Figure 6.8: Percentage of MB types generated by the SAPVC-2 algorithms for different standard video sequences. Arbitrary Shaped Pattern Based Video Coding

Figure 6.9: Coding performance comparisons for standard test video sequences using the SAPVC-2, the SAPVC-1, the APVC, the RTPS(11), and the H.264* standard. Arbitrary Shaped Pattern Based Video Coding

Figure 6.10: Coding performance comparisons for standard test video sequences using the SAPVC-2, the SAPVC-1, the APVC, RTPS(11), and the H.264* standard. those sequences have relatively large moving regions which are well represented by large

Figure 6.11: (a) MotherDaughter frame number 13, (b)—(e) reconstructed frames using the H.264*, Fixed-8, RTPS(11), and the APVC respectively, (f)—(i) frame differences (x2) of (b), (c), (d), and (e) respectively with respect to (a). Arbitrary Shaped Pattern Based Video Coding

Table 2.1: Encoding order and display order of frames in a typical GOP 2.2.1.2 Motion Estimation Techniques so the first P-frame must be processed before the first two B-frames.

Figure 2.7: (a) A typical quantisation matrix, (b) DCT coefficients after applying scalar quantisation to Figure 2.6(b) where the value is 16, and (c) DCT coefficients after applying a quantisation matrix on Figure 2.6(b). small number of bits can represent a data, and thus compression is achieved. For compression, the DCT coefficients are quantised by either a scalar quantisation value o quantisation matrix. A quantisation matrix (see Figure 2.7(a)) typically has small values in th top-left elements of the matrix and larger values in the bottom-right, so retaining lov frequency information at the expense of higher frequencies (see the difference between Figur 2.7(b) and (c)). Using the quantisation matrix however, is very computationally expensivi especially when the coding control mechanism is used to adjust the bit rate. After applyins quantisation, the magnitude of data is reduced (see Figure 2.7(b)). As a result, a relatively emall number of hits can renresent a data and this camnrescion is achieved quantisation, the magnitude of data is reduced (see Figure 2.7(b)). As a result, a relatively

Figure 2.9: (a) Zigzag scanning order, (b) coefficients derived by zigzag scanning on Figure 2.7(b), and (c) variable length codes and zero run length codes from zigzag scanned data using the H.263 recommended VLC look-up table.

Table 2.2: Exp-Golomb code Exponential Golomb (Exp-Golomb) [32] VLC code which is popular because its

Table 2.3: Comparison of digital video coding standards MPEG-1 and MPEG-2 2.3 Block Based VLBR Video Coding

Table 2.4 Various video frame formats used in digital video coding standards While ISO MPEG video/audio standards focused upon generic coding applications, namely the Very Low Bit Rate Video Coding: A Review

Table 2.6: Quantisation step size used in the H.264 CODEC The VLBR coding especially in videophone applications, it is common for the coefficients of a block to be zero after motion estimation/compensation, DCT and quantisation. A method

Table 3.1: Actual number and percentage of different MBs generated by the Fixed-8 algorithm using the quadrant based MB classification The rule for classifying the type of the k-th MB is defined as:

Table 3.2: Pattern numbers with their corresponding RMB frequency (%) and VLC using a large number of standard and non-standard video sequences

Table 3.3: Pattern number with their corresponding RMB frequency (%) and VLC using a large number of standard and non-standard video sequences where the second best similar pattern might replace the best similar pattern in favour of overall coding efficiency and the most frequent pattern has a null code

Table 3.4: The average length of each pattern identification code using the VLC (Huffman) and the enhance VLC schemes Figure 3.10 shows the comparative coding performance using the Fixed-8 and the Fixed- 32 algorithms for the Miss America sequence. The figure confirms that the PVC algorithm

la Ab), = lal, - la A |, where a and b are Boolean variables Lemma 3.1: |a ~ 4], =\a|, +|b], -2|a\d|, where a and b are Boolean variables. Table 3.5: Truth table proof of relations la ~ d, = la|, + \D|, - 2\a Ad, an

Table 3.6: Percentage of RMBs with respect to the total MBs and average moving region size of RMBs generated by the Fixed-32 algorithm using PISM and PESM * The term PISM (also PESM) refers to both a similarity metric, and the Fixed-32 algorithm using the same similarity metric, that are readily identifiable based on the context. Theorem 3.1: PISM and PESM are equivalent in identifying the best pattern for any moving

Table 3.7: Comparative computational complexity of the Fixed-32 algorithm using the PISM and PESM against H.264* using (3.6)-(3.8) with the percentages of CRMBs, and AMBs

Table 4.1: Relevance threshold values Tr(7min) fora dynamically created CPC having a lower bound of min and an upper bound of 7max (77min) patterns size of a CPC. As with Tr(7min), because of the infinite number of possible real coordinates,

Table 4.2: PSNR values for standard sequences using the RTPS(2) and the RTPS(11) algorithms. Although the RTPS algorithm allows us to use any arbitrary large size of pattern codebook

Table 5.1: Percentage of RMBs for standard sequences using the EVPS(32,75%), Fixed-32, Fixed-|3- Opt(8)|, EVPS(8,75%), and the Fixed-8 algorithms with respect to the Fixed-|3-Opt(32)| algorithm

Table 6.1: Percentage of different MB types (defined in Section 3.3) generated by the APVC algorithm 6.3 The Scalable APVC (SAPVC) Algorithm

Table 6.2: Rules for classifying MB types in SAPVC MBs are classified into five types in pattern generation phase in SAPVC such as, SMB Arbitrary Shaped Pattern Based Video Coding

descriptionView Paper arrow_downwardDownload

FPGA DESIGN FOR H.264/AVC ENCODER

by International Journal of Computer Science, Engineering and Applications (IJCSEA)

In this paper, we describe an FPGA H.264/AVC encoder architecture performing at real-time. To reduce the critical path length and to increase throughput, the encoder uses a parallel and pipeline architecture and all modules have been... more

descriptionView Paper arrow_downwardDownload

Joint video coding and statistical multiplexing for broadcasting over DVB-H channels

by mehdi rezaei

2008, Multimedia, IEEE Transactions on

A novel joint video encoding and statistical multiplexing (StatMux) method for broadcasting over digital video broadcasting for handhelds (DVB-H) channels is proposed to improve the quality of encoded video and to decrease the end-to-end... more

descriptionView Paper arrow_downwardDownload

M-LTW: A fast and efficient intra video codec

by otoniel lopez

2008

Intra-video coding is a common way to process video material for applications like professional video editing systems, digital cinema, video surveillance applications, multi-spectral satellite imaging, HQ video delivery, etc. Most... more

descriptionView Paper arrow_downwardDownload

Implementation of Constrained 1-Bit Transform Based Motion Estimation Algorithm with An FPGA Based Architecture

by Günhan Dündar and

2007, 2007 IEEE 15th Signal Processing and Communications Applications

Bu çalışmada blok uyumlama temelli hareket kestirimi işleminin gerçek zamanlı yapılabilmesini sağlamak amacıyla FPGA tabanlı özgün bir donanım önerilmiştir. Tasarlanan sistem genel amaçlı bir FPGA yongası üzerinde çok az yer kapladığı... more

descriptionView Paper arrow_downwardDownload

Very low bit rate video coding using 3-D models

by P. Eisert

2001

Figure 2.1: Structure of a model-based codec.

3. Model-Based Estimation of 3-D Motion and Deformation Figure 3.15: Upper row: video frames of an artificial head object illuminated in different ways, lower row: corresponding model frames after lighting adaptation with 2nd order reflection model.

Figure 3.16: Average reconstruction quality for different sizes of the reflectance map. 3. Model-Based Estimation of 3-D Motion and Deformation

This depth map is obtained by rendering the 3-D object model into a z-buffer. Setting x. = [0 0 OJ and f, = f, = 1 reduces (3.9) to the well-known motior constraint in [LH84, NS85, WKPS87, LRF93]. This corresponds to a linearization o the motion model around the camera origin instead of linearization at the object center However, the linearization error for the reduced case is slightly larger. This is illustratec in Fig. 3.1 where the average absolute displacement error for different rotation angles is plotted for the object shown in Fig. 3.4 b). ] Especially for larger rotation angles significan differences in accuracy are noticeable. At the same time, neither the number of equation: nor the number of unknowns and thus the complexity of solving the system of equations as described in the next section is affected by the improved linearization method.

Figure 3.1: Average absolute displacement error in pixels caused by the linearization of the motion model; a) error of (3.9) for rotation around the x- and y-axes (measured in degrees), b) corresponding errors for the simplified motion model of [LH84].

The optical flow constraint (3.11) can be set up for any pixel in the image. However since the two unknown components d, and d, of the displacement vector vary from pixe to pixel, the flow constraint is underdetermined and cannot be solved directly without additional information. This information is taken directly from the explicit 2-D motior model (3.9) that describes the displacement vector d for all image points containing the object as a function of the 6 unknown rigid body parameters. Inserting (3.9) into the optical flow constraint (3.11) leads to a set of linear equations for the unknowns AR, AR,, AR,, At,, Aty, and At,

The rigid body motion estimator of the previous section estimates the motion parame- ters of an object between two successive frames of a video sequence. To overcome the restrictions from the linearization of small motion between these frames, the algorithm is embedded in an analysis-synthesis loop [Koc93, LRF93] as depicted in Fig. 3.2. Instead of Figure 3.2: Analysis-synthesis loop of the motion estimation framework.

Figure 3.3: Hierarchical estimation scheme. is applied to a synthetically rendered image and the new camera frame. At the begin- ning of the estimation, the motion parameter set corresponding to the previous frame [, is used to render an estimate I for the current camera frame I. The relative motion between these frames is determined as described above using the shape information from the 3-D model. Then, the 3-D model is moved according to the new motion parameter set and a new motion compensated frame J is rendered that is now much closer to the camera frame Jy than the previous estimate. This procedure is performed iteratively, each time reducing the remaining estimation errors. The whole scheme can be regarded as an opti- mization procedure t hat finds the best match between the synthetic frame and the camera frame. Due to the use of a synthetic frame that directly corresponds to the 3-D model providing the algorit hm with shape information, error accumulation over time is avoided, since remaining errors are removed in the following iteration steps. Thus, external forces from discrete image eatures as, e.g., used in [DM96] to avoid error accumulation can be

Figure 3.5: Absolute error of the estimated motion parameter averaged over 100 frames of a synthetic sequence with additive Gaussian noise with standard deviation of 0, 2, 5, 10, 20, and 50, respectively. 3. Model-Based Estimation of 3-D Motion and Deformation

Figure 3.6: Absolute error of the estimated motion parameter averaged over 100 frames using a 3-D model with distorted vertex positions. 3.1. Rigid Body Motion Estimation

Figure 3.7: Absolute error of the estimated motion parameter averaged over 100 frames using a model with distorted vertex positions and texture extracted from an initial frame.

parameter estimation of ARz, ARy, ARz, Atz, At,, Atz, and Ak; performed using those two frames Figure 3.8: Estimation of radial lens distortion and rigid body motion. with the radius r computed as follows

This displacement field is quite similar to the one for rigid body motion (3.9) but contains the additional unknowns Af, Ag, and Ax«;. As a result, by combining (3.22) with the into (3.18) leads, after first order Taylor series expansion, to the following displacement Inserting (3.20) together with the replacement for the 3-D coordinates

Many algorithms in computer vision assume that the internal parameters of the camera are known. Therefore, camera calibration is performed in an initial step to determine focal length, aspect ratio, and radial lens distortion of the camera. A calibration test object as shown in Fig. 3.9 is captured with the camera. Since the geometry of the test Figure 3.9: Two different calibration test objects used in the experiments of this thesis.

Figure 3.10: Average absolute error of the estimated internal and external camera parameters for test objects of varying accuracy. Aspect ratio a and height angle Ypeignt are specified relative to their mean @ = 4 and Pneight = 0.5, respectively. 3.2. Combined Estimation of Motion and Camera Parameters

Figure 3.11: Frames of the test object from different viewing positions. 3. Model-Based Estimation of 3-D Motion and Deformation

Figure 3.12: Standard deviation of the internal camera parameters for different numbers of frames used simultaneously in the estimation.

Figure 3.13: Illumination estimation and compensation scheme.

and sparse and thus efficient algorithms for the inversion are available |GL83}. qairections that ao not alrectly colncide with the elements OF tne Map. For the estimation of reflectance maps, (3.33) is solved for the unknown map entries R°(nz,Ny). For nearest neighbor interpolation, the pixels can easily be classified according to their normals and each entry of the reflectance map is estimated independently of the others. Higher order interpolation leads to a coupling of the unknowns and a linear syste of equations has to be solved. Fortunately, the resulting matrix to be inverted is bandec and sparse and thus efficient algorithms for the inversion are available [GL83].

Figure 3.17: Absolute motion estimation error averaged over 50 frames. The two cases for simple motion estimation assuming ambient illumination and combined motion and illumination estimation (Lambert reflection) are shown. performed but now both motion and illumination parameters are estimated alternately in the hierarchical analysis-synthesis loop shown in Fig. 3.2. From 50 calibrated video frames with constant illumination, the motion parameters are estimated and compared with the original ones. Fig. 3.17 shows the average absolute error of the six rigid body

Figure 3.18: a) topology of the triangular B-spline patches, b) individually adapted triangle mesh used for rendering and motion estimation, c) control points specifying the shape of the textured surface.

Figure 3.19: Modeling of eye movements, a) with blending, b) without blending.

hair if viewed from a certain viewing direction are colored appropriately; the remainin; texels are set to be transparent. The color information for this additional plane can b: directly taken from a video frame since the internal camera parameters specifying th perspective projection are known from the virtual scene. This way, the silhouette of th hair is accurately recovered if the viewing direction is not changed too much. Other wise, multiple planes can be used to obtain a natural appearance for arbitrary viewin, directions. Fig. 3.21 shows some results obtained with this technique.

Figure 3.21: Different facial expressions generated by deforming the surface of a 3-D head model.

Figure 3.22: Transformations from FAPs to image points.

Figure 3.23: a) undistorted model frame, b) synthetic frame with additive noise, c) synthetic frame with distorted control point positions.

Figure 3.24: Absolute error of the estimated motion parameters averaged over 100 frames of a synthetic sequence with additive Gaussian noise (standard deviation 0, 2, 5, 10, 20, and 50, respectively).

3. Model-Based Estimation of 3-D Motion and Deformation Figure 3.25: Absolute error of the estimated motion parameters averaged over 100 frames. The frames of the sequence are rendered from a 3-D model with distorted control point positions.

Figure 4.2: a) Three frames of the sequence Peter, b) corresponding synthetic views. 4.2. Analysis and Synthesis of Head-and-Shoulder Sequences

Figure 4.3: PSNR for the sequence Peter for the two cases that, first, only global motion of the head and, second, both global and local motion is estimated. 4. Model-Based Coding of Head-and-Shoulder Sequences

Figure 4.4: a) Three frames of the sequence Eckehard, b) corresponding synthetic views.

Figure 4.5: a) Three frames of sequence Akiyo, b) corresponding synthetic views.

Figure 4.6: Temporal evolution of Camp and cq; for sequence Illumination. 4. Model-Based Coding of Head-and-Shoulder Sequences

4. Model-Based Coding of Head-and-Shoulder Sequences Figure 4.7: a) Three frames of the sequence Illumination recorded under varying illumination conditions, b) corresponding synthetic views for the 8 parameter Lambertian illumination model.

Figure 4.8: Extracted model of the clapper board.

Figure 4.9: a) Three frames of sequence Clapper Board, b) corresponding synthetic views. the entire object. For that purpose, the clapper board is segmented from the backgrounc and the obtained region is automatically triangulated as depicted in Fig. 4.8. The joint o the clapper board that allows opening it is specified manually. Five animation parameter: determine pose and shape of the object: translation in three directions, rotation in the image plane and opening of the clapper board. These five parameters are estimated ir addition to the FAPs of the head model using two independent motion estimators. Fig. 4.‘ shows three frames of the sequence together with the corresponding model frames createc by rendering the two objects with the estimated parameters. The leftmost image in this figure represents the decoded frame with the worst PSNR (27.2 dB) in the entire sequence

Figure 4.10: PSNR for sequence Clapper Board for the two cases that, first, only facial motion and, second, animation parameters for both the face and the additional clapper board are estimated.

Figure 4.11: Reconstruction quality in PSNR over bit-rate needed for encoding the animation parameters; a) sequence Peter, b) sequence Eckehard. AVLLLS All ALLvbilitsovuiwe CULL VeiViv MMV tELS VLA UUU MA YVLEL ULL ULLAL. In a first experiment, the 18 FAPs estimated as described in Section 4.2 are encoded at 25 fps for the two sequences Peter and Eckehard. The bitstream is decoded and the heac model is animated according to the quantized values synthesizing the sequence again PSNR is measured for this synthetic sequence for different bit-rates obtained by varying the quantizer step size. Fig. 4.11 shows the rate-distortion plots obtained for the model. based codec. Due to model failures in the 3-D head model, the curves saturate at highe1 bit-rates and converge to the quality obtained for unquantized FAPs. The bit-rate needec to come close to that point is around 1 kbps. When using a frame skip of 2 and encoding

Figure 4.12: Reconstruction quality in PSNR over bit-rate needed for encoding the FAPs with a frame skip of 2; a) sequence Peter, b) sequence Eckehard. 4. Model-Based Coding of Head-and-Shoulder Sequences

Figure 4.13: Reconstruction quality in PSNR over bit-rate needed for encoding the illumina- tion parameters. A bit-rate of zero corresponds to no illumination compensation; a) sequence Illumination, b) sequence Peter with a fairly constant illumination situation.

Figure 4.14: upper row: original video sequence. From this sequence, FAPs are estimated and used to synthesize new sequences with different 3-D models. 4. Model-Based Coding of Head-and-Shoulder Sequences

Figure 5.1: Structure of the model-aided coder. Traditional block-based MCP from the previous decoded frame is extended by prediction from the current model frame. 5.2. Rate-Constrained Coder Control

Fig. 5.2 shows the rate-distortion curves obtained for the sequence Peter. Significant gains Figure 5.2: Rate-distortion plot for the sequence Peter. .3.1 Model-Aided Coding of Head-and-Shoulder Sequences

The image on the left hand side of Fig. 5.3 is decoded and reconstructed from the TMN-10 decoder, while the frame on the right hand side originates from the model-aided coder. Both frames require about the same number of bits and are taken from sequences that are approximately encoded at the same average bit-rate using a DCT quantizer parameter of 31 for the TMN-10 and a value of 25 for the model-aided coder.

Figure 5.4: Rate-distortion plot for the sequence Eckehard.

For the first two experiments, an explicit head model created from a 3-D laser scan was available. This is not the case for the next video sequence. Fig. 5.6 shows results for the sequence Akiyo. A generic head model is roughly adapted to the first frame of the sequence (see Section 4.2) and model frames as shown in Fig. 4.5 are rendered from the approximate object model. Even if only an inaccurate 3-D model is used for the generation of the model frames, bit-rate savings of the model-aided codec are still about

35 % at low bit-rates. The quality of the reconstructed frames is shown in Fig. 5.7. The Figure 5.6: Rate-distortion plot for the sequence Akiyo.

Figure 5.8: Model and camera frame 21 of the sequence Akiyo with enlargement of the image region around the mouth. The impact of the rate-constrained coder control on the reference frame selection is exemplarily shown for one frame of the sequence Akiyo. Fig. 5.8 depicts the model- based prediction of frame 21 generated by rendering the 3-D head model. As can be seen in the enlargement of the region around the mouth, the interior of the mouth is not correctly modeled since this information was not available during model acquisition. The correctly modeled since this information was not available during model acquisition. The

Figure 5.9: Frame 21 of the sequence Akiyo. Macroblocks shown have been selected from the model frame. The grey parts of the image are predicted from the previous frame, or they are encoded in INTRA mode. in Fig. 5.9. Fig. 5.9 shows all macroblocks that are predicted from the model frame, while the macroblocks predicted from the previous frame or encoded in INTRA mode are grey. The mouth is not predicted from the model frame thus avoiding the prediction error coding from the generic teeth in Fig. 5.8. The same applies to macroblocks containing the object silhouette, since the background of the scene is not correctly modeled.

Figure 5.11: Frame 66 of the IJlumination sequence coded at about the same bit-rate using TMN-10, and the MAC coder without and with illumination compensation, a) TMN-10 (33.1 dB PSNR, 6764 bits), b) MAC without illumination compensation (33.8 dB PSNR, 5239 bits), c) MAC with illumination compensation (35.1 dB PSNR, 5160 bits).

Figure 5.12: Temporal evolution of PSNR and bit-rate for the sequence Clapper Board. Both coders use a DCT quantizer parameter of 31; a) PSNR, b) bit-rate.

Figure 5.13: Rate-distortion plot for sequence Clapper Board; a) PSNR averaged over the entire sequence, b) PSNR measured for the first 60 frames that show the clapper board.

Figure 5.14: Frame 54 of the sequence Clapper Board. a) Model frame with clapper board missing; b) MAC, 36.1 dB, 2900 bits; c) TMN-10 with same bit-rate as MAC, 32.7 dB, 3200 bits; d) TMN-10 with same PSNR as MAC, 36.0 dB, 5800 bits.

Figure A.1: Scene model and coordinate systems. The virtual scene that approximates the real world consists of multiple objects, light sources, and a camera as shown in Fig. A.1. The objects in the scene can either be rigid

Figure A.2: Components of the camera model. A.3.1 Perspective Projection

Figure A.3: Perspective projection and coordinate systems. TH84, Tek95].

Table 3.1: Experimental setup for the synthetic test sequence. RGB color images as shown in Fig. 3.4 c). Then, the motion parameters are estimated for each frame. Four different levels of resolution are used by the hierarchical algorithm, starting with images of size 44 x 36 and ending with CIF resolution. Since the correct motion parameters are known for the synthetic sequence, the accuracy of the motion estimation is determined by comparing the estimates with the correct values. Din 2K aghacra tha ahanlatoan nrrnar far panh matinn naramatnar arnarag~an nrvaor all INN

Table 3.3: Experimental setup for the synthetic test sequence. ~ — ~*~ To quantify the accuracy of the algorithm, a synthetic sequence is rendered using the calibration test object shown on the left hand side of Fig. 3.9. The experimental setup is specified in Table 3.3. Different sets of internal and external camera parameters are chosen to synthesize the frames. To simulate manufacturing errors of the calibration body, the 2-D positions of the white points on the test object planes are distorted by adding zero mean normally distributed noise with standard deviation o to the point locations before rendering the 3-D model. Given the synthetic frames corresponding to different before rendering the 3-D model. Given the synthetic frames corresponding to different

The performance of the different approaches are compared in Table 3.4. The param- eters of all reflection and lighting models are estimated for all frames and the synthetic frames are shaded appropriately. The resulting model frames are compared with the orig- inal frames in terms of PSNR measured inside the object area. The black background is excluded from the measurement. Table 3.4 shows the resulting PSNR for the differ- ent approaches averaged over all frames. Even for the Lambert reflection model, the ent approaches averaged over all frames. Even for the Lambert reflection model, the

tion is then estimated as described in Section 3.3.1 and again compared with the original vectors. As shown in the first row of Table 3.5, two iterations in the analysis-synthesis loop are sufficient to compensate also for large changes in the illuminant direction. After the third iteration, no further changes are observable. If the maximum angle between the correct illuminant direction and the initial value is less than 45° or 30°, respectively, the convergence is even faster as illustrated in the lower two rows of Table 3.5. convergence is even faster as illustrated in the lower two rows of Table 3.5.

In the next experiment, a real world video sequence consisting of 50 frames is recorded. The illumination setup is held constant during the acquisition but the head object rotates around its y-axis leading to varying shading effects at the object surface. Fig. 3.4 a) depicts the first frame of this sequence that is used to estimate the illumination differences between the synthetic and the real scene. The resulting illumination parameters are ther used to compensate all following frames of the video sequence. PSNR between the model and the camera frame is computed for the facial area of the images. Table 3.6 shows the average reconstruction quality for the different approaches. Even for this experiment. Table 3.6: Average reconstruction quality of the model frames shaded with different reflection models. the average reconstruction quality for the different approaches. Even for this experiment,

Table 4.1: Video sequences used for the experiments. 4. Model-Based Coding of Head-and-Shoulder Sequences

Table 4.2: Reconstruction quality of the sequence Illumination for different lighting models. circumvent this problem, the estimation of illumination parameters is performed in two steps. First, a set of parameters is determined describing differences between the texture map of the model acquired under controlled illumination and the first camera frame. Given the estimated values, the shading effects of the camera frame are removed by applying the inverse of the relation (3.27) before mapping the camera frame onto the model. Since the corrected texture map corresponds to a homogeneously illuminated frame, the estimation of photometric properties remains linear and is performed following the description in Section 3.3.

Table 4.3: Bit-rate needed to encode the FAPs at a reconstruction quality less than 0.1 dB inferior to the unquantized case. The results are given for a frame skip of 0 and 2, respectively.

Table 5.1: Percentage of macroblocks that are selected from the model frame for different quantizer values and video sequences. as many bits. upper right image shows frame 150 encoded with the model- aided coder, while the upper left image corresponds to the TMN-10 coder at the same bit-rate. At the lower right of Fig. 5.7, a frame from the TMN-10 coder is shown that has the same PSNR as the upper model-aided frame. Even though the PSNR is the same, the subjective quality of the reconstructed frame from the model-aided coder is superior since facial features are reproduced more accurately and with fewer artifacts. The dit fference is even more striking when viewing motion sequences. Finally, the lower left image is encoded with TMN-10 to yield the same subjective quality as the model-aided coder; 1] as many bits. MN-10 requires about twice

Table 5.2: Relative frequency of the 4 modes SKIP, INT] model-aided coder. Two different quantizer parameters (31 and 15) Peter (PE), Eckehard (EK), Akiyo (AK), and Illumination (IL). 1 ER, INT) ER-4V, and INTRA for the are chosen for the sequences Che lower three rows of this table depict the coding mode frequency for the case where the model frame is chosen for pre- diction.

of both global and local motion. In the following table, the facial animation parameters of the head model described in Section 3.4.1 are listed. The names of the FAPs correspond directly to the parameters defined in MPI translational g of both global EG-4 |[MPG99] but the actual parameter numbers do not. In addition, 3 obal head movement parameters are added to obtain a uniform description and local motion.

Table C.2: Absolute error of the estimated motion parameters averaged over 100 frames. The frames of the sequence are rendered from a 3-D model with distorted control point positions. All errors are measured in millimeters except for parameters 43-45 which represent angular errors given in degrees. C. Numerical Estimation Results

Institute, University of Erlangen-Nuremberg

descriptionView Paper arrow_downwardDownload

Power Dissipation of the Network-on-Chip in Multi-Processor System-on-Chip Dedicated for Video Coding Applications

by Dragomir Milojevic

2009, Journal of Signal Processing Systems

In the near future, small electronic hand-held devices will be equipped with digital cameras capable of acquiring high resolution images (HDTV) at real-time rates, resulting in video streams of dozens of megabytes per second. The... more

descriptionView Paper arrow_downwardDownload

A Fast Method for H.263 to H.264 Intra Smart-Transcoding

by Alessandro Andreoli

2000

Abstract—The present paper,concerns,the reduction,of com- putational burden,on the estimation of prediction modes,in Intra frame,from,H.263 to H.264 AVC streams,transcoding,process. Scope of this work,is to achieve the possibility of... more

Figure 1. H.263+ Intra prediction process. Intra prediction is defined in Annex I of H.263. The standard proposes three different direction modes: vertical, horizontal and DC. Only 16x16 partitioning is available and prediction is realized in DCT domain over 8x8 blocks. One of the three possible directions is used for each macroblock except for the first macroblock of the frame; in fact horizontal prediction mode is mandatory for this macroblock. For each macroblock the same direction mode is used for all four 8x8 luma blocks and also for both chrominance blocks. DCT coefficients of the block on the left, on the top or both are used as predictor. The entire process of prediction is carry out internally the current macroblock as in Figure 1.

Now we focus on the implementation of the FPDT architec- ture: to realize simulations and tests, a real-time oriented archi- tecture is implemented. An open-source thread library is used As in Figure 3, we can see that H.263+ decoder starts witl thread 1, while thread 2 is dedicated to H.264 decoder process A critical section is shared by the two threads according tc producer-consumer law. These threads run separately sharing only data in a buffer. This buffer is composed by an array o: informative structures: each one contains useful informatior about encoding process (see Figure 4). Buffer size depend: Figure 4. Buffer informative structure.

Figure 5. 16x16 transcoding algorithm. Figure 5 shows the block diagram of the transcoder algo- rithm used for processing the 16x16 Intra macroblocks. All these operations are executed in “Choice I Pred” block of Figure 3. All modes employed are extracted during the “Modes extraction” process of the H.263+ input bitstream.

Figure 7. True positive percentage for seven sequences.

Figure 8. PSNR for “Mother and daughter” sequence with QP=28. Regarding quality measure, we calculate the PSNR value between the original YUV sequences and the those produced by fast and full transcoding procedure. Significant results are shown in Figure 8. Notice that the difference between the two

Figure 10. PSNR for “Stefan” sequence with QP=40. Figure 9. PSNR for “Bridge far” sequence with QP=23.

Figure 12. Transcoding CPU cycles for seven different sequences. Figure 11. Transcoding average bitrate for seven different sequences.

COMPARISON OF INTRA CODING IN H. 263+ AND H.264 STANDARDS pression architecture, both standards share the same logical procedures, implemented in their own specific manner. General procedure for determining which mode has to be used is to calculate the cost for all available modes and take that with minimum cost. Typically, cost is evaluated by SAD (Sum of Absolute Difference): prediction mode giving minimum SAD is the best choice in term of quality and bitrate. Table I

PREDICTION ACCURACY IN A TESTED MACROBLOCK WITH 120 AS AC FIXED THRESHOLD A. Intra 16x16 blocks

H.263+ MACROBLOCK MODES AND COST Table III Threshold evaluation is a key feature in the algorithm. Initial analysis on macroblock costs is presented in Table III. In the third column there are present the percentage of correspondent modes computed by the two standards. In this case H.263+ and H.264 Intra decision blocks give the same both Intra prediction mode, so there is no need for recalculation of the best mode. A fixed threshold can be used to reveal if the correct macroblock gives the same prediction mode, but it does not take in account the frame features. In the second column the average cost of macroblocks is presented and it represents the characteristics of the video sequences. An adaptive threshold following macroblock cost can give better results. In detail, 40% of Intra modes are the same for both standards, so the threshold value has to be set in ord er to accept about 40% of H.263+ modes. The following formula is proposed:

CORRELATION BETWEEN EXACT AND ACCEPTED 16X16 MODES Table IV

descriptionView Paper arrow_downwardDownload

Dynamic Scene Change Detection in Video Coding

by yusra ahmed

2020, International Journal of Engineering Transactions B: Applications

Video compression has become a source of different research studies. It is necessary in order to address channel bandwidth limitations and growing video demand, including digital libraries and streaming media delivery via the Internet. A... more

descriptionView Paper arrow_downwardDownload

MULTI ROBOT PATH PLANNING ALGORITHMS: A SURVEY

by PRJ Publication and

2011, Prj publication

To find the optimal path by interacting with multiple robots is the main research area in field of robotics. The task is to find the global optimal path with a minimum amount of computation time. Path planning has numerous application... more

Uaelelia lil d LU Stull div NICU UL d SlLUUY ts UlLSVELOeU TIVO d He W ULVIstUll UL Ue CiVILUTIIE TIL, Now suppose that we want to find the minimum of J((1) where (1[(/’(.e. [lis a p-dimensional vector of real numbers),and we do not have measurements or an analytical description of the gradient ||J( (J). BFOA mimics the four principal mechanisms observed in a real bacterial system: chemotaxis, swarming, reproduction, and elimination-dispersal to solve this non-gradient optimization problem. A virtual bacterium is actually one trial solution (also called a search- agent) that moves on the functional surface (see Figure | [13]) to locate the global optimum. Figure 1: A bacterial swarm on a multi-modal objective function surface [13]

Figure 2: Outline of ACO for RPP of a mobile robot [14] International Journal of Computer Science and Engineering Research and Development (IJCSERD), ISS} 2248-9363 (Print), ISSN 2248-9371 (Online) Volume 1, Number 1, April-June (2011)

International Journal of Computer Science and Engineering Research and Development (IJCSERD), ISSN 2248-9363 (Print), ISSN 2248-9371 (Online) Volume 1, Number 1, April-June (2011)

As depicted in Figure 4, the heuristic calculation can be calculated as shown below:

descriptionView Paper arrow_downwardDownload

Video Coding

Related Topics