Automatic Audio Morp'hing

2015

This paper presents the author’s experience in developing a software called “morph tunnel” for morphing sounds in real time in the frame of the MaxMSP environment, as well as its application for the composition and performance of a work, “timbre tunnel” for 6 musicians, dancer (optional) and live electronics. The main idea is to create a spectral – and temporal – “tunnel” through a method of pattern matching between overtones, where the user – or a dancer – can navigate in real time between spectra and time in a continuous way.

The Spectral Toolbox is a suite of analysis–resynthesis programs that locate relevant partials of a sound and allow them to be resynthesized at specified frequencies. This enables a variety of routines including spectral mappings (changing all partials of a source sound to fixed destination frequencies), spectral morphing (continuously interpolating between the partials of a source sound and those of a destination sound), and what we call Dynamic Tonality (a novel way of organizing the relationship between a family of tunings and a set of related spectra). A complete application called the TransFormSynth provides a concrete implementation of Dynamic Tonality.

2005

Sound morphing is one of the successful synthesis technologies for recent computer music, and several studies have been reported on this subject. We have proposed a sound morphing algorithm based on a sinusoidal model. In this paper, a correspondence search algorithm is improved, which is core technology of morphing based on a sinusoidal model. Then wider applications of the algorithm are introduced. It can be used as concatenation of two sounds if morphing interval is as short as 100 msec. It is also used as stretch and squeeze of a sound, if the algorithm is applied to two copies from the same sample. Moreover, adding a musical expression to an instrumental sound are discussed.

A system for morphing the brightness of two sounds independently from their other perceptual or acoustic attributes was coded, based on the Spectral Modelling Synthesis additive/residual model. A Multidimensional Scaling analysis of listening test responses showed that the brightness control was perceptually independent from the other controls used to adjust the morphed sound. A Timbre Morpher, providing perceptually meaningful controls for additional timbral attributes, can now be considered for further work.

2005

We present a method for morphing between smooth spectral magnitude envelopes of speech. An important element of our method is the notion of audio flow, which is inspired by similar notions of optical flow computed between images in computer vision applications. Audio flow defines the correspondence between two smooth spectral magnitude envelopes, and encodes the formant shifting that occurs from one sound to another. We present several algorithms for the automatic computation of audio flow from a small 20 second corpus of speech. In addition, we present an algorithm for morphing smoothly between any two spectral magnitude envelopes, given the computed audio flow between them.

Sound morphing figures prominently as one of the most interesting sound transformation techniques due to its enormous creative potential. Most authors pose the problem of morphing sounds using perceptual requirements, but hardly ever evaluate their results mainly because perceptual evaluations are cumbersome and costly, and there are no standard objective evaluation criteria established for sound morphing. In this work we propose a formal evaluation procedure for sound morphing algorithms following three criteria, namely correspondence, intermediateness, and smoothness. The adoption of the proposed evaluation framework will help formalize the results towards more perceptually relevant morphed sounds.

2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03).

An elastic spectral distance measure based on a F0 adaptive pitch synchronous spectral estimation and selective elimination of periodicity interferences, that was developed for a high-quality speech modification procedure STRAIGHT [1], is introduced to provide a basis for auditory morphing. The proposed measure is implemented on a low dimensional piecewise bilinear time-frequency mapping between the target and the original speech representations. A preliminary test results of morphing emotional speech samples indicated that proposed procedure provides perceptually monotonic and high-quality interpolation and extrapolation of CD quality speech samples.

The aim of sound morphing is to obtain a sound that falls perceptually between two (or more) sounds. Ideally, we want to morph perceptually relevant features of sounds and be able to independently manipulate them. In this work we present a method to obtain perceptually intermediate spectral envelopes guided by highlevel spectral shape descriptors and a technique that employs evolutionary computation to independently manipulate the timbral features captured by the descriptors. High-level descriptors are measures of the acoustic correlates of salient timbre dimensions derived from perceptual studies, such that the manipulation of the descriptors corresponds to potentially interesting timbral variations.

Proc. ICMC, 2010

The aim of sound morphing is to obtain a result that falls perceptually between two (or more) sounds. In order to do this, we should be able to morph perceptually relevant features of sounds instead of blindly interpolating the parameters of a model. In this work we present automatic timbral morphing techniques applied to musical instrument sounds using high-level descriptors as features. High-level descriptors are measures of the acoustic correlates of salient timbre dimensions derived from perceptual studies, so that matching the descriptors becomes the goal itself to render the results more perceptually meaningful.

2009

In this paper, a new morphing algorithm for transient sounds is introduced. Input sounds are first projected onto orthogonal bases from which intermediate or novel sounds can be generated. The proposed algorithm uses a shift invariant version of discrete wavelet transform and the singular value decomposition (SVD) to represent the input sound signals over a set of orthogonal bases. Interpolation is carried out between the weight vectors from the SVD to produce a new weight vector used for synthesising a new set of wavelet coefficients. The morphed sound is generated by taking the inverse discrete wavelet transform of the combined weighted bases. The proposed algorithm not only generates a range of new sounds, but also represents the input sounds in a more compact fashion.