580 California St., Suite 400
San Francisco, CA, 94104
Key research themes
1. How are digital audio effects designed and modeled to manipulate and enhance musical sound?
This research area investigates the design, modeling, and implementation of digital audio effects (DAFx) that manipulate musical sound for creative and production purposes. It focuses on algorithmic development, real-time processing, and the application of machine learning for black-box and parametric modeling of audio effects such as equalization, amplification, and non-linear distortions. These advances are significant for music production, mixing, live performance, and the emulation of analog hardware in digital environments.
Key finding: This comprehensive collection reveals that the field of digital audio effects is undergoing a shift towards integrating machine learning methods, as shown by contributions applying deep neural networks for modeling analog... Read more
Key finding: This work categorizes audio effects based on amplitude and phase modulation techniques using delay lines, explaining the signal processing foundations underlying effects such as vibrato, flanging, chorus, and rotary speaker... Read more
Key finding: This seminal review consolidates diverse digital audio equalization techniques from classic to cutting-edge approaches within a unified mathematical framework, emphasizing parametric and shelving filters, graphic equalizers,... Read more
Key finding: This paper presents innovations to parametric multichannel audio coding through the Immersive Sound-field Rendition (ISR) system, focusing on improvements like phase compensated down-mixing and blind up-mixing schemes for low... Read more
2. What advances enable automatic detection and removal of non-speech vocal sounds (e.g., breath sounds) in digital audio recordings?
This important subfield addresses the challenge of identifying and eliminating unwanted non-speech vocal sounds such as breaths that may detract from the clarity of vocal recordings in music, broadcasting, and speech processing. The focus encompasses algorithmic approaches employing signal processing and deep learning methods to automate this laborious process. Effective solutions improve both production efficiency and audio quality, with notable progress shown in attention U-Net architectures that combine accuracy with reduced computational complexity.
Key finding: This study introduces a parameter-efficient deep learning model leveraging an attention U-Net architecture for automatic detection and eradication of breath sounds in vocal recordings. Trained on a unique DAPS-derived dataset... Read more
Key finding: The presented modulation-based effects models inherently manage dynamic changes in audio signal properties suited to vocal manipulation. The insights into phase and amplitude modulation mechanisms inform the design of... Read more
3. How are digital sound synthesis and digital sound reconstruction advancing in generating realistic audio?
This theme covers algorithmic innovations in synthesizing sounds digitally, particularly through physical modeling and digital reconstruction techniques. The goal is to produce realistic and high-fidelity synthetic musical or acoustic sounds. Physical modeling simulates vibrating physical structures, enabling expressive synthesis linked to instrument mechanics. Digital sound reconstruction (DSR) techniques aim to overcome limitations of traditional loudspeaker designs, especially in low-frequency domains, using array configurations and shutter gate mechanisms to enhance sound pressure output. These approaches enable real-time synthesis and new sound generation paradigms critical for musical instrument digital interfaces and audio reproduction devices.
Key finding: This article systematically presents physical modeling methods for digital sound synthesis, transforming partial differential equation-based models of vibrating structures (such as strings and drums) into discrete-time... Read more
Key finding: The paper introduces Advanced Digital Sound Reconstruction (ADSR), which improves upon classical DSR by incorporating shutter gates and redirection mechanisms to boost sound pressure level particularly in mid-to-low frequency... Read more