Perceptual audio quality assessment

We present a system for content-based retrieval of perceptually similar sound events in audio documents ('sound spotting', using a query by example. The system consists of three discrete stages: a front-end for feature extraction, a self-organizing map, and a pattern matching unit. Our paper introduces the approach, describes the separate modules and discusses some preliminary results and future research.

To facilitate better consistency between programs and stations, ITU, EBU and ARIB have investigated the standardization of broadcast loudness. This paper examines some consequences of a global loudness standard with regard to metering and control at the Ingest, Production and Transmission stages. Findings are reported from the latest research into mono, stereo and multichannel loudness measurement of real-world broadcast sounds. The improvements achieved by the new loudness models are quantified against previous level descriptors, such as, for example, PPM and Leq(A). Besides from reducing consumer annoyance with jumping levels, less engineering time needs being spent per audio stream. This too, is important because digital broadcast means a significant proliferation of the number of channels and the number of platforms. Each platform, such as TV, radio, internet, podcast, and other personal entertainment systems, has its own requirements for dynamic range, frequency range and speec...

The ITU-R BS.1770 multichannel loudness algorithm performs a sum of channel energies with weighting coefficients based on azimuth and elevation angles of arrival of the audio signal. In its current version, these coefficients were estimated based on binaural summation gains and not on subjective directional loudness. Also, the algorithm lacks directional weights for wider elevation angles (jfj � 30�). A listening test with broadband stimuli was conducted to collect subjective data on directional effects. The results were used to calculate a new set of directional weights. A modified version of the loudness algorithm with these estimated weights was tested against its benchmark using the collected data, and using program material rendered to reproduction systems with different loudspeaker configurations. The modified algorithm performed better than the benchmark, particularly with reproduction systems with more loudspeakers positioned out of the horizontal plane.

A systematic review of typical biases encountered in modern audio quality listening tests is presented. The following three types of bias are discussed in more detail: bias due to affective judgments, response mapping bias, and interface bias. In addition, a potential bias due to perceptually nonlinear graphic scales is discussed. A number of recommendations aiming to reduce the aforementioned biases are provided, including an in-depth discussion of direct and indirect anchoring techniques.

This paper provides complementary data to the review of biases in audio quality listening tests by Zieliński et al. (2008) [1]. The paper presents selected illustrations of range equalizing bias, centering bias, stimulus spacing bias, contraction bias, and bias due to nonlinear properties of assessment scale. The illustrations are given in graphical form and respective discussions of biases using empirical data obtained by various researchers over the period of the past 15 years. The presented collection of illustrations along with the discussion may help the experimenters to identify potential biases affecting their data and avoid typical pitfalls in reporting the outcomes of the listening tests.

Basic perceptual quality of coded audio material is commonly evaluated using ITU-R BS-1534 MUSHRA (Multi Stimulus with Hidden Reference and Anchors) listening tests. MUSHRA guidelines call for experienced listeners. However, the majority of consumers using the final product are no expert-listeners. Also the degree of expertise in a listening test may vary amongst listeners in the same laboratory. It would be useful to know how the audio quality evaluation differs between trained and untrained listeners and how training and actual tests should be designed in order to be as reliable as possible. To investigate the rating differences between experts and non-experts, we performed MUSHRA listening tests with 13 experienced and 11 inexperienced listeners using 5 speech and audio codecs delivering a wide range of basic audio quality. Except for the hidden reference, absolute ratings of non-experts were consistently higher than those of experts. However, rank order only rarely changed between experts and non-experts. For lower quality values, confidence intervals were significantly larger for non-experts than for experts. Experienced listeners set more than twice as many loops as non-experts, compared more often between codecs and listened to high quality codecs for a longer duration than non-experts.

There are a range of different methods for comparing or measuring the similarity between environmental sound effects. These methods can be used as objective evaluation techniques, to evaluate the effectiveness of a sound synthesis method by assessing the similarity between synthesised sounds and recorded samples. We propose to evaluate a number of different synthesis objective evaluation metrics, by using the different distance metrics as fitness functions within a resynthesis algorithm. A recorded sample is used as a target sound, and the resynthesis is intended to produce a set of synthesis parameters that will synthesise a sound as close to the recorded sample as possible, within the restrictions of the synthesis model. The recorded samples are excerpts of selections from a sound effects library, and the results are evaluated through a subjective listening test. Results show that one of the objective function performs significantly worse than several others. Only one method had a...

This material was originally intended as part of the article "The Loudness War: Background, Speculation and Recommendations" [1] but was removed for reasons of scope and to keep that article to a manageable length.) In this paper, I briefly review a variety of metrics for quantifying the loudness and dynamic spread of audio recordings. This review was motivated by the need for objective ways of measuring the effects of hypercompression as used in the loudness war.

The current ITU's standard for objective assessment of audio quality, Perceptual Evaluation of Audio Quality (PEAQ), has some shortcomings that prevent its reliable use for a number of codification conditions and some kind of signals. The paper aims to improve the PEAQ performance through the following proposals: 1-modifications in the manner the signals are submitted to the assessment; 2-improvement of existing Model Output Variables (MOVs); 3-creation of new MOVs; 4-determination of a better architecture for the neural network that maps the MOVs into a single estimate for the subjective score. The results are compared to those ones achieved by PEAQ.

Over the last decades, the simulation of musical instruments by digital means has become an important part of modern music production and live performance. Since the first release of the Kemper Profiling Amplifier (KPA) in 2011, guitarists have been able to create and store a nearly unlimited number of “digital fingerprints” of amplifier and cabinet setups for live performances and studio productions. However, whether listeners can discriminate between the sounds of the KPA and the original amplifier remains unclear. Thus, we constructed a listening test based on musical examples from both sound sources. In a first approach, the psychoacoustic analysis using mel-frequency cepstrum coefficients (MFCCs) revealed a high degree of timbre similarity between the two sound sources. In a second step, a listening test with N = 177 showed that the overall discrimination performance was d’ = .34, which was a rather small difference (0.0 ≤ d’ ≤ 0.74). A weak relationship between the degree of g...

This paper presents a study undertaken to evaluate user ratings on auditory feedback of sound source selection within a multi-track auditory environment where sound placement is controlled by a gesture control system. Selection confirmation is presented to the participants via changes to the audio mixture over the stereo loudspeakers or feedback over a single ear bluetooth headset. Overall five different methods are compared and results of our study are presented. A second task in the study was given to evaluate a preselection method to help find sound sources before selection, the participant altered a width control of the pre-selection that was heard in the bluetooth headset. Results indicate a specific value irrespective of genre that the pre-selection should be set to whilst the selection confirmation can be perceived to be dependant on genre and instrumentation.

Perceptual listening tests are commonplace in audio research and a vital form of evaluation. While a large number of tools exist to run such tests, many feature just one test type, are platform dependent, run on proprietary software, or require considerable configuration and programming. Using Web Audio, the Web Audio Evaluation Tool (WAET) addresses these concerns by having one toolbox which can be configured to run many different tests, perform it through a web browser and without needing proprietary software or computer programming knowledge. In this paper the role of the Web Audio API in giving WAET key functionalities are shown. The paper also highlights less common features, available to web based tools, such as easy remote testing environment and in-browser analytics.

This paper presents dual sensor based management system for television viewing at home environment in compliance with BS1770 loudness measurement standard aim of the research was mainly to address the issue of sudden increase in loudness during commercials in between TV programmes. Same approach works for any well. The scope of this research reported here investigate, develop an adaptive loudness control system to create a pleasurable TV viewing environment and to algorithm functionality by simulations. The novel made in paper are the design and the proof of functional efficacy of the loudness management algorithms using the measurements made through the simple instrumentation set in compliance with BS1770 loudness measurement

While the LUFS standard was originally developed for broadcast applications, it offers a convenient means of calibrating program material stimuli to an equal loudness level, while remaining in a multichannel format. However, this calibration is based on an absolute sound pressure level of 60dBA, the preferred listening level when watching television. Levels used in analytical listening and perceptual experiments tend to be significantly higher. This disparity may affect the accuracy of the Leq(RLB) weighting filter employed in LUFS meters. To address this issue, the development of the LUFS standard is examined to assess its suitability for the task. The findings suggest that a compromise between analytical listening and loudness matching in perceptual experiments requires careful consideration of experimental variables.

In this age of DTV systems which allow wide dynamic range, we easily find media content which is not within the confort zone of most of the listeners. With the advent of various object loudness measurement techniques and compliance standar ds like CALM, loudness control has become even more important for content produc ers. The goal of loudness control should be to achieve consistent loudness acro ss different programs as well within a program so that users don't feel need to reach for remote control. Traditionally, peak normalization has often been used as the technique to control loudness of programs. But just peak normalization c annot properly normalise loudness based on modern metrics. Broadly, loudness Co ntrol techniques can be divided into two Categories.

Evaluating audio source separation algorithms means rat- ing the quality or intelligibility of separated source sig- nals. While objective criteria fail to account for all audi- tory phenomena so far, precise subjective ratings can be obtained by means of listening tests. In practice, the accu- racy and the reproducibility of these tests depend on sev- eral design issues. In this

In the last 20 year there has been an increasing need for an objective method for eval- uating audio from a perceptual point of view. Perceptual encoding and its prevalence in popular audio distribution models highlights the demand for software that is able to estimate the quality without having to organize costly and time-consuming listening experiments which have been the main means of evaluating perceptual audio quality so far. In addition to this timbre has been the subject of many publications recently that have contributed to understanding the connection between di↵erent metrics to quantify it and their perceptual analog. MFCCs have recently been shown to have a close rela- tion with perception of sound and the Echonest Analyzer API has been proven to be particularly successful at measuring timbre in MIR (Music Information Retrieval) tasks and it efficacy and ease of use make it a perfect candidate for exploring timbre.
The project proposes a new approach to objective audio quality evaluation in which, in order to find computationally the perceptual di↵erences between two tracks, timbre features are retrieved using the Echonest Analyzer, a perceptually based audio analysis service, and MFCC, a perceptually relevant feature set used in speech recognition. A distance measure derived from speech recognition research, Dynamic Time Warping, is used in conjunction with the Euclidean distance of two vectors representing the first four statistical moments of the features derived, are used to acquire a 6-dimensional feature set detailing dissimilarity between two tracks. These are then used with labels obtained in listening tests in the training of a system that uses K-Nearest Neighbour regression to predict quality. An experiment is designed to gather data for the train- ing and validation of the system. The quality prediction are found to correlate with subjective ratings and compared with the PEAQ standard, which is found to perform better. Finally considerations are made about the verification process and about how this research can be taken forward.

There is a consensus among many in the audio industry that recorded music has grown increasingly
compressed over the past few decades. Some industry professionals are concerned that this
compression often results in poor audio quality with little dynamic range. Although some algorithms have
been proposed for calculating dynamic range, we have not been able to find any studies suggesting that
any of these metrics accurately represent any perceptual dimension of the measured sound. In this
paper, we review the various proposed algorithms and compare their results with the results of a listening
test. We show that none of the tested metrics accurately predict the perceived dynamic range of a
musical track, but we identify some potential directions for future work.

The current ITU's standard for objective assessment of audio quality, Perceptual Evaluation of Audio Quality (PEAQ), has some shortcomings that prevent its reliable use for a number of codification conditions and some kind of signals. The paper aims to improve the PEAQ performance through the following proposals: 1-modifications in the manner the signals are submitted to the assessment; 2-improvement of existing Model Output Variables (MOVs); 3-creation of new MOVs; 4-determination of a better architecture for the neural network that maps the MOVs into a single estimate for the subjective score. The results are compared to those ones achieved by PEAQ.

Modern audio mastering procedures include selectively equalisation of specific frequency bands for the tonal enhancement of the unmastered material. This process is mostly based on music scores'  or listening information, like the musical key or the mode of the  audio material. Based on these, mastering engineers apply enhancement or attenuation at specific frequency bands, relative to the frequencies that correspond to the musical's key notes. In this work, we present an adaptive and automated equalisation system for the aforementioned mastering procedure based on a novel method for fundamental frequency  estimation. Moreover, PEAQ analysis and listening tests were implemented to evaluate the  system's performance.