Subjective Evaluation of Ultra-high Definition (UHD) Videos

2010

The upcoming High-Definition format for video display provides high-quality content, especially when displayed on adapted devices. When combined with video coding techniques such as MPEG-4 AVC/H.264, the transmission of High-Definition video content on broadcast networks becomes possible. Nonetheless, transmitting and decoding such video content is a real challenge. Therefore, intermediate formats based on lower frame resolutions or interlaced coding are still provided to address targets with limited resources. Using these formats, the final video quality depends on the postprocessing tools employed at the receiver to up-sample and de-interlace these streams. In this paper, we compare the full-HD format to three possible scenarios to generate a full-HD stream from intermediate formats. We present the results of subjective tests that compare the visual quality of each scenario when using the same bitrate. The results show that using the same bitrate, the videos generated from lower-resolution formats reach similar quality compared to the full-HD videos.

IEEE Transactions on Broadcasting, 2008

This paper proposes a novel method for the assessment of picture quality, called Triple Stimulus Continuous Evaluation Scale (TSCES), to allow the direct comparison of different HDTV formats. The method uses an upper picture quality anchor and a lower picture quality anchor with defined impairments. The HDTV format under test is evaluated in a subjective comparison with the upper and lower anchors. The method utilizes three displays in a particular vertical arrangement. In an initial series of tests with the novel method, the HDTV formats 1080p/50, 1080i/25, and 720p/50 were compared at various bit-rates and with seven different content types on three identical 1920 x 1080 pixel displays. It was found that the new method provided stable and consistent results. The method was tested with 1080p/50, 1080i/25, and 720p/50 HDTV images that had been coded with H.264/AVC High profile. The result of the assessment was that the progressive HDTV formats clearly outperformed the interlaced HDTV format. A system chain proposal is given for future media production and delivery to take advantage of this outcome. Recommendations for future research conclude the paper.

2012

ABSTRACT High Efficiency Video Coding (HEVC) is the latest attempt by ISO/MPEG and ITU-T/VCEG to define the next generation compression standard beyond H. 264/MPEG-4 Part 10 AVC. One of the major goals of HEVC is to provide efficient compression for resolutions beyond HDTV. However, the subjective evaluations that led to the selection of technologies were bound to HDTV resolution.

The main objective of this paper is to verify test methodologies for assessment high dynamic range (HDR) video. To achieve this, a next generation HDR monitor by Dolby Laboratories was used to display professionally produced HDR content, including some scenes from recently released movies. Two complementary approaches for subjective assessment of HDR video were then designed and carried out at the EBU and EPFL premises. Results obtained from both evaluations were highly correlated which show they offer a good degree of reliability and reproducibility in different situations. Analysis of the scores in both cases also show good confidence intervals for each point under test. Finally, they could demonstrate that an increase in terms of quality of experience can be expected from the conventional level of 100 nits to HDR/high brightness at 4'000 nits, with intermediate improvements at 400 and 1'000 nits.

IEEE Access, 2020

The paper presents a series of three new video quality model standards for the assessment of sequences of up to UHD/4K resolution. They were developed in a competition within the International Telecommunication Union (ITU-T), Study Group 12, in collaboration with the Video Quality Experts Group (VQEG), over a period of more than two years. A large video quality test set with a total of 26 individual databases was created, with 13 used for training and 13 for validation and selection of the winning models. For each database, video quality laboratory tests were run with at least 24 subjects each. The 5-point Absolute Category Rating (ACR) scale was used for rating, calculating Mean Opinion Scores (MOS) as groundtruth. To represent today's commonly applied HTTP-based adaptive streaming context, the test sequences comprise a variety of encoding settings, bitrates, resolutions and framerates for the three codecs H.264/AVC, H.265/HEVC and VP9, applied to a wide range of source sequences of around 8 s duration. Processing was carried out with an FFmpeg-based processing chain developed specifically for the competition, and via upload and encoding through exemplary online streaming services. The resulting data represents the largest, lab-test-based dataset used for video quality model development to date, with a total of around 5,000 test sequences. The paper addresses the three models ultimately standardized in the P.1204 Recommendation series, resulting in different model types and for different applications: (i) Rec. P.1204.3, no-reference bitstream-based, with access to encoded bitstream information; (ii) P.1204.4, pixel-based, using information from the reference and the processed video, and (iii) P.1204.5, no-reference hybrid, using both bitstreamand pixel-information without knowledge of the reference. The paper outlines the development process and provides holistic details about the statistical evaluation, test databases, model algorithms and validation results, as well as a performance comparison with state-of-the-art models. INDEX TERMS Bitstream, full reference, http adaptive streaming (HAS), hybrid, pixel-based, QoE, reduced reference, video quality.

2007

The impact of H.264 artefacts on subjective quality is still under investigation [1]. Contrary to existing approaches, this paper considers the impact on perceived quality of real H.264 artefacts in HDTV videos. A method for the design of spatio-temporal classification is proposed. This classification is used to locally distort sequences with the H.264 codec. Class-generated sequences are then subjectively assessed in order to evaluate the importance of each spatiotemporal class. An attempt to find a relation between local and global quality loss is then presented and discussed, along with an annoyance function model.

2012 5th International Congress on Image and Signal Processing, 2012

Advancements in the video processing area have been proliferated by services that require low delay. Such services involve applications being offered at various temporal and spatial resolutions. It necessitates to study the impacts of related video coding conditions upon perceptual quality. But most of studies concerned with quality assessment of videos affected by coding distortions lack in variety of spatio-temporal resolutions. This paper presents a work done on quality assessment of videos encoded by state-of-the-art H.264/AVC standard at different bitrates and frame rates. Overall, 120 test scenarios for video sequences having different spatial and temporal spectral information were studied. The used coded bistreams in this work and the corresponding subjective assessment scores have been made public for the research community to facilitate further studies. TABLE I DESCRIPTION OF USED VIDEOS Spatial Resolutions Video Sequences Bit rates(kbps)

Proceedings of the Proceedings of the 7th Mathematics, Science, and Computer Science Education International Seminar, MSCEIS 2019, 12 October 2019, Bandung, West Java, Indonesia, 2020

This study aims to carry out an objective assessment by evaluating the encoding performance of the new standard compared to its predecessor H.264/AVC. In addition, to produce a comprehensive investigation, we compared several resolutions (1080p, 720p, and 480p) with a combination of CRF and encoder presets. To find out the performance of both techniques we used several test parameters such as encoding duration, compression ratio, bitrate, MSE, and PSNR. The results showed that H.264 was faster in terms of the time needed to carry out the encoding process. However the compression ratio and bitrate of H.265 was better with a difference of 38.5% and 52.7% respectively. Finally, if the user wants to prioritize better quality video output without having to sacrifice a lot of compression time and small file size, then our recommendation is to use H.265 and by HD resolution configuration (720p) with medium presets and 18 CRF value.

2015 IEEE International Symposium on Multimedia (ISM), 2015

The new video coding standard, MPEG-H Part 2 High Efficiency Video Coding (HEVC) or H.265, was developed to be roughly twice as efficient as H.264/AVC-meaning H.265/HEVC could deliver the same quality as H.264/AVC using roughly half the bitrate. In this paper we describe a subjective experiment designed to test this claim. We present an experiment using 20 different 1080p 29.97 fps scenes and 12 impairment levels spanning MPEG-2, H.264/AVC and H.265/HEVC. Additionally we compare the results obtained from the subjective assessment to quality estimates from two objective metrics: VQM and PSNR. Our subjective results show that H.265/HEVC can deliver the same quality at half the bitrate compared to H.264/AVC and can perform better at one quarter the bitrate compared to MPEG-2 in many, but not all, situations. For all 20 scenes coded with H.265/HEVC at 4 Mbps mean opinion scores span 38% of the subjective scale, which indicates the importance of scene selection. Objective quality estimations of HEVC have a low correlation with subjective results (0.60 for VQM, 0.64 for PSNR).

Applied Sciences

The paper presents the results of subjective and objective quality assessments of H.264-, H.265-, and VP9-encoded video. Most of the literature is devoted to subjective quality assessment in well-defined laboratory circumstances. However, the end users usually watch the films in their home environments, which may be different from the conditions recommended for laboratory measurements. This may cause significant differences in the quality assessment scores. Thus, the aim of the research is to show the impact of environmental conditions on the video quality perceived by the user. The subjective assessment was made in two different environments: in the laboratory and in users’ homes, where people often watch movies on their laptops. The video signal was assessed by young viewers who were not experts in the field of quality assessment. The tests were performed taking into account different image resolutions and different bit rates. The research showed strong correlations between the ob...