Perceptually-motivated audio morphing: brightness

KANSEI Engineering International, 2004

A psychophysical investigation of timbre was undertaken with the intent of deriving quantitative results that could be useful in developing a better human interface to control effects processing for expressiveness in music. Multidimensional perceptual scaling analysis was performed for a set of musical timbres. Two listening experiments were performed in order to determine the most salient perceptual attributes. These two experiments provide data enabling KANSEI engineering for these effects. A commercially available effects processor generated the six stimuli used in these experiments. In the first experiment, listeners were asked to give ratings of the relative difference between two stimuli without regard to any property contributing to that difference. In the second experiment , the listeners were asked to give ratings of each stimulus with respect to a particular property. Dissimilarity ratings of all pair-wise comparisons of the six effects were submitted to INdividual Differences SCALing (INDSCAL) analysis in order to derive a multidimensional perceptual space for the stimuli. Three semantic components were derived using Principal Components Analysis (PCA) of property ratings for the same set of six effects. Multiple Regression Analysis (MRA) was used to relate the perceptual dimensions and semantic components, the results of which provide a control structure for a commercially available effects processor. The study revealed a direct mapping between three perceptual dimensions and three semantic component scales, those being labeled as "sharp-dull," "diffusecompact," and "thick-thin" (in Japanese, surudoi-nibui, hirogotta-kojinmarishita, and usui-atsui).

Proceedings of the 14th International Audio Mostly Conference: A Journey in Sound

It is not uncommon to hear musicians and audio engineers speak of warmth and brightness when describing analog technologies such as vintage mixing consoles, multitrack tape machines, and valve compressors. What is perhaps less common, is hearing this term used in association with retro digital technology. A question exists as to how much the low bit rate and low-grade conversion quality contribute to the overall brightness or warmth of a sound when processed with audio effects simulating early sampling technologies. These two dimensions of timbre are notoriously difficult to define and more importantly, measure. We present a subjective user study of brightness and warmth, where a series of audio examples are processed with different audio effects. 26 participants rated the perceived level of brightness and warmth of various instrumental sequences for 5 different audio effects including bit depth reduction, compression and equalisation. Results show that 8 bit reduction tends to increase brightness and decrease warmth whereas 12 bit reduction tends to do the opposite, although this is very much dependent on the instrument. Interestingly, the most significant brightness changes, due to bit reduction, were obtained for bass sounds. For comparison purposes, instrument phrases were also processed with both an analogue compressor and an equalisation plugin to see if any subjective difference was noticed when simulating sonic characteristics that might be associated with warmth. Greater significance was observed when the sound excerpts were processed with the plugins being used to simulate the effects of bit depth reduction. CCS Concepts: • Information systems → Speech / audio search; • Applied computing → Sound and music computing; Media arts; • Computing methodologies → Perception; • Software and its engineering → Semantics.

2001

A real-time synthesis engine which models and predicts the timbre of acoustic instruments based on perceptual features extracted from an audio stream is presented. The thesis describes the modeling sequence including the analysis of natural sounds, the inference step that finds the mapping between control and output parameters, the timbre prediction step, and the sound synthesis. The system enables applications such as cross-synthesis, pitch shifting or compression of acoustic instruments, and timbre morphing between instrument families. It is fully implemented in the Max/MSP environment. The Perceptual Synthesis Engine was developed for the Hyperviolin as a novel, generic and perceptually meaningful synthesis technique for non-discretely pitched instruments. Advisor: Tod Machover Title: Professor of Music and Media Perceptual Synthesis Engine: An Audio-Driven Timbre Generator

An algorithm that achieves independent modification of two low-level features that are correlated with the auditory perceptions of brightness and warmth was implemented. The perceptual validity of the algorithm was tested through a series of listening tests in order to examine whether the low-level modification was indeed perceived as independent and to investigate the influence of the fundamental frequency on the perceived modification. A Multidimensional Scaling analysis (MDS) on listener responses to pairwise dissimilarity comparisons accompanied by a verbal elicitation experiment, examined the perceptual significance and independence of the two low-level features chosen. This is a first step for the future development of a perceptually based control of an additive synthesizer.

An algorithm that achieves independent modification of two low-level features that are correlated with the auditory perceptions of brightness and warmth was implemented. The perceptual validity of the algorithm was tested through a series of listening tests in order to examine whether the low-level modification was indeed perceived as independent and to investigate the influence of the fundamental frequency on the perceived modification. A Multidimensional Scaling analysis (MDS) on listener responses to pairwise dissimilarity comparisons accompanied by a verbal elicitation experiment, examined the perceptual significance and independence of the two low-level features chosen. This is a first step for the future development of a perceptually based control of an additive synthesizer.

International Conference on Acoustics, Speech, and Signal Processing, 1996

This paper describes techniques to automatically morph from one sound to another. Audio morphing is accomplished by representing the sound in a multi-dimensional space that is warped or modified to produce a desired result. The multi-dimensional space encodes the spectral shape and pitch on orthogonal axes. After matching components of the sound, a morph smoothly interpolates the amplitudes to describe

This paper presents an implementation of a near real-time timbre morphing signal processing system, designed to facilitate an element of 'liveness' and unpredictability in a musical installation. The timbre morpher is a hybrid analysis and synthesis technique based on Spectral Modeling Synthesis (an additive and subtractive modeling technique). The musical installation forms an interactive soundtrack in response to the series of Rosso Luana marble sculptures Shapes in the Clouds, I, II, III, IV & V by artist Peter Randall-Page, exhibited at the Peninsula Arts Gallery in Devon, UK, from 1 February to 29 March 2014. The timbre morphing system is used to transform live input captured at each sculpture with a discrete microphone array, by morphing towards noisy source signals that have been associated with each sculpture as part of a predetermined musical structure. The resulting morphed audio is then fed-back to the gallery via a five-channel speaker array. Visitors are encouraged to walk freely through the installation and interact with the sound world, creating unique audio morphs based on their own movements, voices, and incidental sounds.

Inspired by a formal analogy between the two dimensional version of Laplace's partial differential equation and the two dimensions (time and frequency) of a spectrogram, this paper explores a variety of density and kernelbased techniques that can smoothly connect (morph between) two functions. When the functions represent audio spectra, this presents a solution to the problem of audio morphing, providing a concrete way of adjusting the overtones of a sound while smoothly interpolating between existing sounds. The approach can be applied to both interpolation morphing (where the morph connects two different sounds over some specified duration) and to repetitive morphing (where a series of sounds are generated, each containing progressively more features of one sound and fewer of the other). When successful, the timbre of the sounds is changed dynamically in a plausible way. A series of sound examples demonstrate the strengths and weaknesses of the approach. * Both sethares@ece.wisc.edu and bucklew@engr.wisc.edu are with the

The goal of sound morphing by feature interpolation is to obtain sounds whose values of features are intermediate between those of the source and target sounds. In order to do this, we should be able to resynthesize sounds that present a set of predefined feature values, a notoriously difficult problem. In this work, we present morphing techniques to obtain hybrid musical instrument sounds whose feature values correspond as close as possible to the ideal interpolated values. When the features capture perceptually relevant information, the morphed sound whose features are interpolated is perceptually intermediate. The features we use are acoustic correlates of salient timbre dimensions derived from perceptual studies, such that sounds whose feature values are intermediate between two would be placed between them in the underlying timbre space. We measure the perceptual impact of the morphed sounds directly by the feature values, using them as an objective measure with which to evaluate the results. Thus we consider that the morphed sounds change perceptually linearly when the corresponding feature values vary linearly.

The aim of sound morphing is to obtain a sound that falls perceptually between two (or more) sounds. Ideally, we want to morph perceptually relevant features of sounds and be able to independently manipulate them. In this work we present a method to obtain perceptually intermediate spectral envelopes guided by highlevel spectral shape descriptors and a technique that employs evolutionary computation to independently manipulate the timbral features captured by the descriptors. High-level descriptors are measures of the acoustic correlates of salient timbre dimensions derived from perceptual studies, such that the manipulation of the descriptors corresponds to potentially interesting timbral variations.