Laplacian Directed Audio Morphing

IEEE Transactions on Audio, Speech, and Language Processing, 2000

Sound morphing is a transformation that gradually blurs the distinction between the source and target sounds. For musical instrument sounds, the morph must operate across timbre dimensions to create the auditory illusion of hybrid musical instruments. The ultimate goal of sound morphing is to perform perceptually linear transitions, which requires an appropriate model to represent the sounds being morphed and an interpolation function to obtain intermediate sounds. Typically, morphing techniques directly interpolate the parameters of the sound model without considering the perceptual impact or evaluating the results. Perceptual evaluations are cumbersome and not always conclusive. In this work, we seek parameters of a sound model that favor linear variation of perceptually motivated temporal and spectral features used to guide the morph towards more perceptually linear results. The requirement of linear variation of feature values gives rise to objective evaluation criteria for sound morphing. We investigate several spectral envelope morphing techniques to determine which spectral representation renders the most linear transformation in the spectral shape feature domain. We found that interpolation of line spectral frequencies gives the most linear spectral envelope morphs. Analogously, we study temporal envelope morphing techniques and we concluded that interpolation of cepstral coefficients results in the most linear temporal envelope morph.

Sound morphing figures prominently as one of the most interesting sound transformation techniques due to its enormous creative potential. Most authors pose the problem of morphing sounds using perceptual requirements, but hardly ever evaluate their results mainly because perceptual evaluations are cumbersome and costly, and there are no standard objective evaluation criteria established for sound morphing. In this work we propose a formal evaluation procedure for sound morphing algorithms following three criteria, namely correspondence, intermediateness, and smoothness. The adoption of the proposed evaluation framework will help formalize the results towards more perceptually relevant morphed sounds.

The aim of sound morphing is to obtain a sound that falls perceptually between two (or more) sounds. Ideally, we want to morph perceptually relevant features of sounds and be able to independently manipulate them. In this work we present a method to obtain perceptually intermediate spectral envelopes guided by highlevel spectral shape descriptors and a technique that employs evolutionary computation to independently manipulate the timbral features captured by the descriptors. High-level descriptors are measures of the acoustic correlates of salient timbre dimensions derived from perceptual studies, such that the manipulation of the descriptors corresponds to potentially interesting timbral variations.

2005

We present a method for morphing between smooth spectral magnitude envelopes of speech. An important element of our method is the notion of audio flow, which is inspired by similar notions of optical flow computed between images in computer vision applications. Audio flow defines the correspondence between two smooth spectral magnitude envelopes, and encodes the formant shifting that occurs from one sound to another. We present several algorithms for the automatic computation of audio flow from a small 20 second corpus of speech. In addition, we present an algorithm for morphing smoothly between any two spectral magnitude envelopes, given the computed audio flow between them.

Cogent Mathematics, 2015

This paper explores a variety of density and kernel-based techniques that can smoothly connect (crossfade or "morph" between) two functions. When the functions represent audio spectra, this provides a concrete way of adjusting the partials of a sound while smoothly interpolating between existing sounds. The approach can be applied to both interpolation-crossfades (where the crossfade connects two different sounds over a specified duration) and to repetitive-crossfades (where a series of sounds are generated, each containing progressively more features of one sound and fewer of the other). The interpolation surface can be thought of as the two dimensions (time and frequency) of a spectrogram, and the kernels can be chosen so as to constrain the surface in a number of desirable ways. When successful, the timbre of the sounds is changed dynamically in a plausible way. A series of sound examples demonstrate the strengths and weaknesses of the approach.

2005

Sound morphing is one of the successful synthesis technologies for recent computer music, and several studies have been reported on this subject. We have proposed a sound morphing algorithm based on a sinusoidal model. In this paper, a correspondence search algorithm is improved, which is core technology of morphing based on a sinusoidal model. Then wider applications of the algorithm are introduced. It can be used as concatenation of two sounds if morphing interval is as short as 100 msec. It is also used as stretch and squeeze of a sound, if the algorithm is applied to two copies from the same sample. Moreover, adding a musical expression to an instrumental sound are discussed.

A system for morphing the brightness of two sounds independently from their other perceptual or acoustic attributes was coded, based on the Spectral Modelling Synthesis additive/residual model. A Multidimensional Scaling analysis of listening test responses showed that the brightness control was perceptually independent from the other controls used to adjust the morphed sound. A Timbre Morpher, providing perceptually meaningful controls for additional timbral attributes, can now be considered for further work.

This paper presents an implementation of a near real-time timbre morphing signal processing system, designed to facilitate an element of 'liveness' and unpredictability in a musical installation. The timbre morpher is a hybrid analysis and synthesis technique based on Spectral Modeling Synthesis (an additive and subtractive modeling technique). The musical installation forms an interactive soundtrack in response to the series of Rosso Luana marble sculptures Shapes in the Clouds, I, II, III, IV & V by artist Peter Randall-Page, exhibited at the Peninsula Arts Gallery in Devon, UK, from 1 February to 29 March 2014. The timbre morphing system is used to transform live input captured at each sculpture with a discrete microphone array, by morphing towards noisy source signals that have been associated with each sculpture as part of a predetermined musical structure. The resulting morphed audio is then fed-back to the gallery via a five-channel speaker array. Visitors are encouraged to walk freely through the installation and interact with the sound world, creating unique audio morphs based on their own movements, voices, and incidental sounds.

2020

This paper presents a system for morphing between audio recordings in a continuous parameter space. The proposed approach combines matrix decompositions used for audio source separation with displacement interpolation enabled by 1D optimal transport. By interpolating the spectral components obtained using nonnegative matrix factorization of the source and target signals, the system allows varying the timbre of a sound in real time, while maintaining its temporal structure. Using harmonic / percussive source separation as a pre-processing step, the system affords more detailed control of the interpolation in perceptually meaningful dimensions.

2001

Abstract When designing audio effects for music processing, we are always aiming at providing higherlevel representations that may somehow fill in the gap between the signal processing world and the end-user. Spectral models in general, and the Sinusoidal plus Residual model in particular, can sometimes offer ways to implement such schemes.