Creating interactive virtual auditory environments

International Journal of Interactive Multimedia and Artificial Intelligence

Simulating propagation of sound and audio rendering can improve the sense of realism and the immersion both in complex acoustic environments and dynamic virtual scenes. In studies of sound auralization, the focus has always been on room acoustics modeling, but most of the same methods are also applicable in the construction of virtual environments such as those developed to facilitate computer gaming, cognitive research, and simulated training scenarios. This paper is a review of state-of-the-art techniques that are based on acoustic principles that apply not only to real rooms but also in 3D virtual environments. The paper also highlights the need to expand the field of immersive sound in a web based browsing environment, because, despite the interest and many benefits, few developments seem to have taken place within this context. Moreover, the paper includes a list of the most effective algorithms used for modelling spatial sound propagation and reports their advantages and disadvantages. Finally, the paper emphasizes in the evaluation of these proposed works.

Virtual Reality (VR) is on the edge of becoming the next major delivery plat- form for digital content. Technological progress has made accessible a variety of new tools, allowing development of the necessary hardware and software for VR interaction. The importance of audio in this process is increasing. Audio can intensify the three dimensional experience and immersion of game players and users in other applications. This research focuses on determining and implementing the necessary ele- ments for a system able to deliver Virtual Auditory Display (VAD). The system structure is adapted to fit a new emerging gaming platform: smart- phones. Developing for mobile platforms requires consideration of many con- straints such as memory, processor load and development components. The result is a real-time dynamic VAD, manageable by mobile devices, and able to trigger spatial auditory perception across the azimuthal plane. In this study it is also shown how the VAD developed following custom imple- mentation of selected Head Related Transfer Function (HRTF), is able to generate azimuthal localization of sound events in VR scenarios.

Proceedings of the 25th annual conference on Computer graphics and interactive techniques - SIGGRAPH '98, 1998

Virtual environment research has focused on interactive image generation and has largely ignored acoustic modeling for spatialization of sound. Yet, realistic auditory cues can complement and enhance visual cues to aid navigation, comprehension, and sense of presence in virtual environments. A primary challenge in acoustic modeling is computation of reverberation paths from sound sources fast enough for real-time auralization. We have developed a system that uses precomputed spatial subdivision and "beam tree" data structures to enable real-time acoustic modeling and auralization in interactive virtual environments. The spatial subdivision is a partition of 3D space into convex polyhedral regions (cells) represented as a cell adjacency graph. A beam tracing algorithm recursively traces pyramidal beams through the spatial subdivision to construct a beam tree data structure representing the regions of space reachable by each potential sequence of transmission and specular reflection events at cell boundaries. From these precomputed data structures, we can generate high-order specular reflection and transmission paths at interactive rates to spatialize fixed sound sources in real-time as the user moves through a virtual environment. Unlike previous acoustic modeling work, our beam tracing method: 1) supports evaluation of reverberation paths at interactive rates, 2) scales to compute highorder reflections and large environments, and 3) extends naturally to compute paths of diffraction and diffuse reflection efficiently. We are using this system to develop interactive applications in which a user experiences a virtual environment immersively via simultaneous auralization and visualization.

Stereoscopic Displays and Virtual Reality Systems III, 1996

Sound represents a largely untapped source of realism in Virtual Environments (VEs). In the real world, sound constantly surrounds us and pulls us into our world. In VEs, sound enhances the immersiveness of the simulation and provides valuable information about the environment. While there has been a growing interest in integrating sound into VE interfaces, current technology has not brought about its widespread use. This, we believe, can be attributed to the lack of proper tools for modeling and rendering the auditory world. We have been investigating some of the problems which we believe are pivotal to the widespread use of sound in VE interfaces. As a result of this work, we have developed the Virtual Audio Server (VAS). VAS is a distributed, real-time spatial sound generation server. It provides high level abstractions for modeling the auditory world and the events which occur in the world. VAS supports both sampled and synthetic sound sources and provides a device independent interface to spatialization hardware. Resource management schemes can easily be integrated into the server to manage the real-time generation of sound. We are currently investigating possible approaches to this important problem.

2002

ABSTRACT High-quality virtual audio scene rendering is a must for emerging virtual and augmented reality applications, for perceptual user interfaces, and sonification of data. Personalization of HRTF is necessary in applications where perceptual realism and correct elevation perception is critical. We describe algorithms for creation of virtual auditory spaces by rendering cues that arise from anatomical scattering, environmental scattering, and dynamical effects.

Proceedings of the 2006 …, 2006

Traditional uses of virtual audio environments tend to focus on perceptually accurate acoustic representations. Though spatialization of sound sources is important, it is necessary to leverage control of the sonic representation when considering musical applications. The proposed framework allows for the creation of perceptually immersive scenes that function as musical instruments. Loudspeakers and microphones are modeled within the scene along with the listener/performer, creating a navigable 3D sonic space where sound sources and sinks process audio according to user-defined spatial mappings.

Audio Mostly 2006, 2006

Auditory authoring is an essential component in the design of virtual environments and describes the process of assigning sounds and voices to objects within a virtual 3D scene. In a broader sense, auditory authoring also includes the definition of dependencies between objects and different object states, as well as time-and user-dependent interactions in dynamic environments. Our system unifies these attributes within so called auditory textures and allows an intuitive design of 3D auditory scenes for varying applications. Furthermore, it takes care of the different perception through auditory channels and provides interactive and easy to use sonification and interaction techniques. In this paper we present the necessary concepts as well as a system for the authoring of 3D virtual auditory environments as they are used in computer games, augmented audio reality and audio-based training simulations for the visually impaired. As applications we especially focus on augmented audio reality and the applications associated with it. In the paper we provide details about the definition of 3D auditory environments along techniques for their authoring and design, as well an overview of the system itself with a discussion of several examples.

The SoundScape Renderer is a versatile software framework for real-time spatial audio rendering. The modular system architecture allows the use of arbitrary rendering methods. Three rendering modules are currently implemented: Wave Field Synthesis, Vector Base Amplitude Panning and Binaural Rendering. After a description of the software architecture, the implementation of the available rendering methods is explained and the graphical user interface is shown as well as the network interface for the remote control of the virtual audio scene. Finally, the Audio Scene Description Format, a system-independent storage file format, is briefly presented.

Proceedings of International …, 2007

We present a method for user-specific audio rendering of a virtual environment that is shared by multiple participants. The technique differs from methods such as amplitude differencing, HRTF filtering, and wave field synthesis. Instead we model virtual microphones within the 3-D scene, each of which captures audio to be rendered to a loudspeaker. Spatialization of sound sources is accomplished via acoustic physical modelling, yet our approach also allows for localized signal processing within the scene. In order to control the flow of sound within the scene, the user has the ability to steer audio in specific directions. This paradigm leads to many novel applications where groups of individuals can share one continuous interactive sonic space.

In this paper we propose a game-based virtual reality platform for evaluation of sonification techniques. We study the task of local-ization of stationary objects in virtual reality using auditory cues. We further explore sonification techniques and compare the performance in this task using the proposed platform. The virtual reality environment is developed using Unity3D (game engine) and an Oculus Rift, a head mounted virtual reality display. Parameter mapping sonification techniques are employed to map the position of the object in virtual space to sound. Hence, the framework defined here constitutes an auditory virtual reality environment. This auditory display interface is subjectively evaluated in stationary object localization task. A statistical analysis of the subjective and objective measures of the listening test is performed resulting in a robust and scientific evaluation of the sonification methods.