APE: Audio Perceptual Evaluation toolbox for MATLAB

Up until now, the only way to measure the sound quality of modern audio coding systems at low bit rates has been to implement elaborate listening tests with experienced human test subjects. Consequently, the idea of substituting the subjective tests by objective, computer based methods has been an ongoing focus of research and development. The result was PEAQ, an algorithm (ITU-R Recommendation BS.1387-1), that uses a number of psycho-acoustical tests combined to give a measure of the quality difference between two instances of a signal (a reference and a test signal). The paper describes a MATLAB PEAQ method implementation and the procedure used to assess it. Finally, PEAQ is used to rate the quality of operation for some very known digital audio editors and audio codecs.

In the last 20 year there has been an increasing need for an objective method for eval- uating audio from a perceptual point of view. Perceptual encoding and its prevalence in popular audio distribution models highlights the demand for software that is able to estimate the quality without having to organize costly and time-consuming listening experiments which have been the main means of evaluating perceptual audio quality so far. In addition to this timbre has been the subject of many publications recently that have contributed to understanding the connection between di↵erent metrics to quantify it and their perceptual analog. MFCCs have recently been shown to have a close rela- tion with perception of sound and the Echonest Analyzer API has been proven to be particularly successful at measuring timbre in MIR (Music Information Retrieval) tasks and it efficacy and ease of use make it a perfect candidate for exploring timbre. The project proposes a new approach to objective audio quality evaluation in which, in order to find computationally the perceptual di↵erences between two tracks, timbre features are retrieved using the Echonest Analyzer, a perceptually based audio analysis service, and MFCC, a perceptually relevant feature set used in speech recognition. A distance measure derived from speech recognition research, Dynamic Time Warping, is used in conjunction with the Euclidean distance of two vectors representing the first four statistical moments of the features derived, are used to acquire a 6-dimensional feature set detailing dissimilarity between two tracks. These are then used with labels obtained in listening tests in the training of a system that uses K-Nearest Neighbour regression to predict quality. An experiment is designed to gather data for the train- ing and validation of the system. The quality prediction are found to correlate with subjective ratings and compared with the PEAQ standard, which is found to perform better. Finally considerations are made about the verification process and about how this research can be taken forward.

Frontiers in psychology, 2024

Introduction: Previous work on audio quality evaluation has demonstrated a developing convergence of the key perceptual attributes underlying judgments of quality, such as timbral, spatial and technical attributes. However, across existing research there remains a limited understanding of the crucial perceptual attributes that inform audio quality evaluation for people with hearing loss, and those who use hearing aids. This is especially the case with music, given the unique problems it presents in contrast to human speech. Method: This paper presents a sensory evaluation study utilising descriptive analysis methods, in which a panel of hearing aid users collaborated, through consensus, to identify the most important perceptual attributes of music audio quality and developed a series of rating scales for future listening tests. Participants (N = 12), with a hearing loss ranging from mild to severe, first completed an online elicitation task, providing single-word terms to describe the audio quality of original and processed music samples; this was completed twice by each participant, once with hearing aids, and once without. Participants were then guided in discussing these raw terms across three focus groups, in which they reduced the term space, identified important perceptual groupings of terms, and developed perceptual attributes from these groups (including rating scales and definitions for each). Results: Findings show that there were seven key perceptual dimensions underlying music audio quality (clarity, harshness, distortion, spaciousness, treble strength, middle strength, and bass strength), alongside a music audio quality attribute and possible alternative frequency balance attributes. Discussion: We outline how these perceptual attributes align with extant literature, how attribute rating instruments might be used in future work, and the importance of better understanding the music listening difficulties of people with varied profiles of hearing loss.

Perceptual evaluation tests where subjects assess certain qualities of different audio fragments are an integral part of audio and music research. These require specialised software, usually custom-made, to collect large amounts of data using meticulously designed interfaces with carefully formulated questions, and play back audio with rapid switching between different samples. New functionality in HTML5 included in the Web Audio API allows for increasingly powerful media applications in a platform independent environment. The advantage of a web application is easy deployment on any platform, without requiring any other application, enabling multiple tests to be easily conducted across locations. In this paper we propose a tool supporting a wide variety of easily configurable, multi-stimulus perceptual audio evaluation tests over the web with multiple test interfaces, pre-and post-test surveys, custom configuration, collection of test metrics and other features. Test design and setup doesn't require programming background, and results are gathered automatically using web friendly formats for easy storing of results on a server.

2014

The present paper deals with the subjective evaluation of audio coding technologies using the “Similarity Rating” psychometric method. Compressed audio excerpts are presented to a group of experienced listeners by pairs of stimuli. Each pair represents a different type of distortion, with compression included. These types of distortion correspond to three psychoacoustic attributes, sharpness, roughness, and fluctuation strength. Listener’s task is to evaluate the degree of similarity between the pairs. The final output is the localization of each measured codec in this psychoacoustic space. Multidimensional scaling is the statistic technique, belonging to Multivariate Statistical methods, which allows processing the results obtained from this method.

2016

This paper evaluates a perceptually motivated objective model for the measurement of “punch” in musical signals. Punch is a perceptual attribute that is often used to characterize music that conveys a sense of dynamic power or weight. A methodology is employed that combines signal separation, onset detection, and low level parameter measurement to produce a perceptually weighted “punch” score. The model is evaluated against subjective scores derived through a forced pairwise comparison listening test using a wide variety of musical stimuli. The model output indicates a high degree of correlation with the subjective scores. Correlation results are also compared to other objective models such as Crest Factor, Inter-Band-Ratio (IBR), Peak-to-Loudness Ratio (PLR), and Loudness Dynamic Range (LDR).

Storage and Retrieval for Image and Video Databases VII, 1998

In analyzing audio material for features useful for extracting content, we must consider the value gained by adapting our analysis algorithms to the analysis processes of the human ear. This aspect with regard to loudness features is thoroughly examined in this paper. The increase in correlation to be gained by such cognitive processing is about 10.

2008

A systematic review of typical biases encountered in modern audio quality listening tests is presented. The following three types of bias are discussed in more detail: bias due to affective judgments, response mapping bias, and interface bias. In addition, a potential bias due to perceptually nonlinear graphic scales is discussed. A number of recommendations aiming to reduce the aforementioned biases are provided, including an in-depth discussion of direct and indirect anchoring techniques.

KANSEI Engineering International, 2004

A psychophysical investigation of timbre was undertaken with the intent of deriving quantitative results that could be useful in developing a better human interface to control effects processing for expressiveness in music. Multidimensional perceptual scaling analysis was performed for a set of musical timbres. Two listening experiments were performed in order to determine the most salient perceptual attributes. These two experiments provide data enabling KANSEI engineering for these effects. A commercially available effects processor generated the six stimuli used in these experiments. In the first experiment, listeners were asked to give ratings of the relative difference between two stimuli without regard to any property contributing to that difference. In the second experiment , the listeners were asked to give ratings of each stimulus with respect to a particular property. Dissimilarity ratings of all pair-wise comparisons of the six effects were submitted to INdividual Differences SCALing (INDSCAL) analysis in order to derive a multidimensional perceptual space for the stimuli. Three semantic components were derived using Principal Components Analysis (PCA) of property ratings for the same set of six effects. Multiple Regression Analysis (MRA) was used to relate the perceptual dimensions and semantic components, the results of which provide a control structure for a commercially available effects processor. The study revealed a direct mapping between three perceptual dimensions and three semantic component scales, those being labeled as "sharp-dull," "diffusecompact," and "thick-thin" (in Japanese, surudoi-nibui, hirogotta-kojinmarishita, and usui-atsui).

Journal of the Audio Engineering Society, 2016

The quality of recorded music is often highly disputed. To gain insight into the dimensions of quality perception, subjective and objective evaluation of musical program material, extracted from commercial CDs, was undertaken. It was observed that perception of audio quality and liking of the music can be affected by separate factors. Familiarity with stimuli affected like ratings while quality ratings were most associated with signal features related to perceived loudness and dynamic range compression. The effect of listener expertise was small. Additionally, the sonic attributes describing quality ratings were gathered and indicate a diverse lexicon relating to timbre, space, defects, and other concepts. The results also suggest that while the perceived quality of popular music may have decreased over recent years, like ratings were unaffected.