Mixed resolution coding of stereoscopic video for Mobile devices

2017 IEEE International Conference on Imaging Systems and Techniques (IST), 2017

This paper presents a mixed resolution stereo video coding model for High Efficiency Video Codec (HEVC). The challenging aspects of mixed resolution video coding are enabling the codec to encode frames with different frame resolution/size and using decoded pictures having different frame resolution/size for referencing. These challenges are further enlarged when implemented using HEVC, since the incoming video frames are subdivided into coding tree units. The ingenuity of the proposed codec's design, is that the information in intermediate frames are down-sampled and yet the frames can retain the original resolution. To enable random access to full resolution decoded frame in the decoded picture buffer as reference frame a downsampled version of the decoded full resolution frame is used. The test video sequences were coded using the proposed codec and standard MV-HEVC. Results show that the proposed codec gives a significantly higher coding performance over the MV-HEVC codec.

2013

The emerging MVC+D standard specifies the coding of Multiview Video plus Depth (MVD) data for enabling advanced 3D video applications. MVC+D specifications define the coding of all views of MVD at equal spatial resolution and apply a conventional MVC technique for coding the multiview texture and the depth independently. This paper presents a modified MVC+D coding scheme, where only the base view is coded at the original resolution whereas dependent views are coded at reduced resolution. To enable interview prediction, the base view is downsampled within the MVC coding loop to provide a relevant reference for dependent views. At the decoder side, the proposed scheme consists of a post-processing scheme which upsamples of the decoded views to their original resolution. The proposed scheme is compared against the original MVC+D scheme and an average of 4% delta bitrate reduction (dBR) in the coded views and 14.5% of dBR in the synthesized views are reported.

2010

Abstract Asymmetric stereoscopic video coding takes advantage of the binocular suppression of the human vision by representing one of the views with a lower quality. This paper describes a subjective quality test with asymmetric stereoscopic video. Different options for achieving compressed mixed-quality and mixed-resolution asymmetric stereo video were studied and compared to symmetric stereo video.

2016

The use of 3D stereoscopic technology with high quality videos can provide visual entertainment to viewers. However, the bandwidth of typical communication channels cannot transmit uncompressed 3D videos, resulting in the need for video quality compression. This paper presents a series of preliminary studies to investigate the subjective perception of uncompressed and compressed video sequences, and proposes the ‘hybrid’ sequencing of uncompressed and compressed content in a single stereoscopic 3D video as an alternative approach for limited bandwidth transmission. However, the hybrid uncompressed/compressed sequencing of stereoscopic 3D video may affect the correlation between the left and right views of the stereoscopic videos required for depth perception, potentially leading to lower Quality of Experience (QoE) of viewers. This paper therefore investigates both the objective and subjective quality evaluation of the proposed hybrid sequencing of stereoscopic video sequences. Init...

ACM Transactions on Applied Perception, 2006

JPEG compression of the left and right components of a stereo image pair is a way to save valuable bandwidth when transmitting stereoscopic images. This paper presents results on the effects of camera-base distance (B) and JPEG coding on overall image quality, perceived depth, perceived sharpness, and perceived eye strain. In the experiment, two stereoscopic still scenes were used, varying in depth (three different camera-base distances: 0, 8, and 12 cm) and compression ratio (4 levels: original, 1:30, 1:40, and 1:60). All levels of compression were applied to both the left and right stereo image, resulting in a 4 × 4 matrix of all possible symmetric and asymmetric coding combinations. The observers were asked to assess image quality, sharpness, depth, and eye strain. Results showed that an increase in JPEG coding had a negative effect on image quality, sharpness, and eye strain, but had no effect on perceived depth. An increase in camera-base distance increased perceived depth and reported eye strain, but had no effect on perceived sharpness. Results on asymmetric and symmetric coding showed that the relationship between perceived image quality and average bit rate is not straightforward. In some cases, image quality ratings of a symmetric coded pair can be higher than for an asymmetric coded pair, even if the averaged bit rate for the symmetric pair is lower, than for the asymmetric pair. Furthermore, sharpness and eye strain correlated highly and medium, respectively, with perceived image quality.

2010

Abstract In mixed-resolution (MR) stereoscopic video, one of the views has a lower resolution compared to the other one, hence providing means for improved compression. The underlying assumption in MR stereoscopic video is that the human visual system fuses the left and right views in such a way that the perceived image quality is close to that of the higher-resolution view.

IEEE Access

This paper presents a high efficiency video codec-based spatial mixed-resolution stereo video codec. The proposed codec applies a frame interleaving algorithm to reorder the stereo video frames into a single stream of monoscopic video. The challenge for mixed-resolution video coding is to enable the codec to encode frames with different frame resolutions. This issue is addressed by superimposing a low-resolution replica of the decoded I-frame on its respective decoded picture, where remaining space of the frame is set to zero. This significantly reduces the computation cost for finding the best match. The proposed codec's reference frames structure is designed to efficiently exploit both temporal and interview correlations. The performance of the proposed codec is assessed using five standard multiview video data sets and benchmarked against that of the anchor and the state-of-the-art techniques. Results show that the proposed codec yields significantly higher coding performance compared with the anchor and state-of-the-art techniques. INDEX TERMS HEVC, mixed-resolution video codec, stereo video coding, low bitrate transmission.

TURKISH JOURNAL OF ELECTRICAL ENGINEERING & COMPUTER SCIENCES, 2014

In 3D perception, the binocular suppression of the human vision, perceiving high quality 3D video in the case of a view in higher quality, can be exploited in asymmetric stereo video coding for efficient 3D rate scaling. Hence, the best stereo rate-visual distortion performance may be gained by asymmetric coding, which is the reduction of the spatial and/or quantization resolution of the low-quality view, while keeping the high-quality view in full resolution. However, how to determine the level of asymmetry and what type of scaling should be chosen are still in question. In this work, we try to assess the overall performance of the scalability options with several test contents. The test videos are encoded at critical bitrates with symmetric options and spatial or signal-to-noise ratio (SNR) asymmetric coding, and are subjectively evaluated in a stereo-polarized projection 3D display system. Two different types of evaluation methodologies are used: the Double-Stimulus Continuous-Quality Scale (DSCQS) and Subjective Evaluation of Stereo Video Quality (SESVIQ). Dense visual tests show that the spatial scaling is generally inferior when compared to SNR scaling, except that high motion scenes and symmetric SNR are more preferable for a higher bitrate. The characteristics of the video content should be taken into consideration for efficient stereo rate scaling.

MDPI Applied Sciences, 2022

Provisioning the stereoscopic 3D (S3D) video transmission services of admissible quality in a wireless environment is an immense challenge for video service providers. Unlike for 2D videos, a widely accepted No-reference objective model for assessing transmitted 3D videos that explores the Human Visual System (HVS) appropriately has not been developed yet. Distortions perceived in 2D and 3D videos are significantly different due to the sophisticated manner in which the HVS handles the dissimilarities between the two different views. In real-time video transmission, viewers only have the distorted or receiver end content of the original video acquired through the communication medium. In this paper, we propose a No-reference quality assessment method that can estimate the quality of a stereoscopic 3D video based on HVS. By evaluating perceptual aspects and correlations of visual binocular impacts in a stereoscopic movie, the approach creates a way for the objective quality measure to assess impairments similarly to a human observer who would experience the similar material. Firstly, the disparity is measured and quantified by the region-based similarity matching algorithm, and then, the magnitude of the edge difference is calculated to delimit the visually perceptible areas of an image. Finally, an objective metric is approximated by extracting these significant perceptual image features. Experimental analysis with standard S3D video datasets demonstrates the lower computational complexity for the video decoder and comparison with the state-of-the-art algorithms shows the efficiency of the proposed approach for 3D video transmission at different quantization (QP 26 and QP 32) and loss rate (1% and 3% packet loss) parameters along with the perceptual distortion features.

ACM Transactions on …, 2006

JPEG compression of the left and right components of a stereo image pair is a way to save valuable bandwidth when transmitting stereoscopic images. This paper presents results on the effects of camera-base distance (B) and JPEG-coding on overall image quality, perceived depth, perceived sharpness and perceived eye-strain. In the experiment, two stereoscopic still scenes were used, varying in depth (three different camera-base distances: 0, 8 and 12 cm) and compression ratio (4 levels: original, 1:30, 1:40 and 1:60). All levels of compression were applied to both the left and right stereo image, resulting in a 4x4 matrix of all possible symmetric and asymmetric coding combinations. The observers were asked to assess image quality, sharpness, depth and eye-strain. Results showed that an increase in JPEG coding had a negative effect on image quality, sharpness and eye-strain but had no effect on perceived depth. An increase in camera-base distance increased perceived depth and reported eye-strain but had no effect on perceived sharpness. Results on asymmetric and symmetric coding showed that the relationship between perceived image quality and average bit-rate is not straightforward. In some cases, image quality ratings of a symmetric coded pair can be higher than for an asymmetric coded pair, even if the averaged bit rate for the symmetric pair is lower, than for the asymmetric pair. Furthermore, sharpness and eye-strain correlated highly and medium, respectively, with perceived image quality.