580 California St., Suite 400
San Francisco, CA, 94104
Key research themes
1. How can audio texture synthesis techniques facilitate real-time, realistic soundscape rendering?
This research area focuses on the development of methods and models for synthesizing audio textures that replicate the statistical and perceptual properties of natural or environmental sounds, enabling real-time rendering in applications such as virtual reality, gaming, and multimedia. The goal is to achieve perceptually convincing, non-repetitive, and spatially accurate soundscapes, where a sound texture is a complex auditory pattern composed of many micro-events, often with foreground and background layers. It matters because realistic soundscapes enhance immersion and user experience in interactive environments, yet require balancing computational efficiency and auditory plausibility.
Key finding: Introduced a two-layer Markov Model and a hierarchical scheme for generating Head-Related Transfer Function filters to efficiently render decorative sound textures in real time, distinguishing foreground sounds from... Read more
Key finding: Presented a two-stage method for synthesizing audio textures from short example clips by analyzing and segmenting the audio into perceptually meaningful building patterns, then generating arbitrarily long audio streams using... Read more
Key finding: Extended auditory display techniques to the sonification of textured images, modeling textures as convolutions of impulses with basic wavelet-like spots analogous to speech synthesis models. Showed that projecting 2D textures... Read more
2. What psychoacoustic models and perceptual principles underpin effective auditory display and sonification of complex acoustic information?
This theme explores the use of psychoacoustic modeling to represent, analyze, and sonify complex audio data, enabling auditory displays that enhance human perception and interpretation of information. Leveraging models of loudness, sharpness, and auditory texture, it seeks to develop perceptually optimal mappings between data features and sound parameters to improve clarity, intuitiveness, and listener comfort in sonification systems. The theme is significant as it bridges auditory perception science with practical sonification design.
Key finding: Proposed the application of psychoacoustical models for loudness, sharpness, roughness, and fluctuation strength to optimize auditory display design, enabling perceptually meaningful auditory graphs that reduce listener... Read more
Key finding: Demonstrated that human listeners identify brief, three-tone melodic sequences based on relative frequency intervals and their Gestalt-like properties rather than absolute frequency content, with recognition robust under... Read more
Key finding: Showed significant interaction between auditory and visual levels of detail in perception of material rendering quality in immersive virtual environments, evidencing that audio and visual stimuli mutually influence material... Read more
3. How can spatial audio reproduction systems and auralization techniques enhance the perception and analysis of complex auditory scenes?
This research area focuses on the development and evaluation of spatial audio systems, multi-channel reproduction, and simulation methods for accurate rendering and analysis of complex auditory scenes. Key aspects include system architectures supporting flexible rendering methods, perceptual evaluation of different spatial configurations, and simulation of room acoustics for immersive experiences. These advancements are crucial for improving auditory display realism, enhancing soundscape perception, and supporting applications in virtual reality, architectural acoustics, and auditory cognition research.
Key finding: Developed a modular software framework (SSR) capable of real-time reproduction of virtual audio scenes using multiple rendering methods including Wave Field Synthesis, Vector Base Amplitude Panning, and binaural rendering.... Read more
Key finding: Provided a comprehensive interdisciplinary overview of sonification, emphasizing auditory displays' unique suitability for monitoring complex, multi-variable, and temporally dynamic data. Notably, it identified spatial audio... Read more
Key finding: Introduced a particle tracing approach for mid- and high-frequency acoustic simulation in 3D scenes that computes finite impulse response filters accounting for frequency-dependent reflections, enabling realistic auralization... Read more
Key finding: Through listening tests comparing 1D, 2D, and 3D loudspeaker arrays with varied source material, established that subjective spatial sound perception depends strongly on the auditory scene content. No single spatial... Read more