Papers by Joachim Gossmann
Scalable metadata environments (MDE): artistically impelled immersive environments for large-scale data exploration
The Engineering Reality of Virtual Reality 2014, 2014
ABSTRACT Scalable Metadata Environments (MDEs) are an artistic approach for designing immersive e... more ABSTRACT Scalable Metadata Environments (MDEs) are an artistic approach for designing immersive environments for large scale data exploration in which users interact with data by forming multiscale patterns that they alternatively disrupt and reform. Developed and prototyped as part of an art-science research collaboration, we define an MDE as a 4D virtual environment structured by quantitative and qualitative metadata describing multidimensional data collections. Entire data sets (e.g.10s of millions of records) can be visualized and sonified at multiple scales and at different levels of detail so they can be explored interactively in real-time within MDEs. They are designed to reflect similarities and differences in the underlying data or metadata such that patterns can be visually/aurally sorted in an exploratory fashion by an observer who is not familiar with the details of the mapping from data to visual, auditory or dynamic attributes. While many approaches for visual and auditory data mining exist, MDEs are distinct in that they utilize qualitative and quantitative data and metadata to construct multiple interrelated conceptual coordinate systems. These "regions" function as conceptual lattices for scalable auditory and visual representations within virtual environments computationally driven by multi-GPU CUDA-enabled fluid dyamics systems.
Musical Interface to Audiovisual Corpora of Arbitrary Instruments
We present an instrument for audio-visual performance that allows to recombine sounds from a coll... more We present an instrument for audio-visual performance that allows to recombine sounds from a collection of sampled media through concatenative synthesis. A three-dimensional distribution derived from feature-analysis becomes accessible through a theremin-inspired interface, allowing the player to shift from exploration and intuitive navigation toward embodied performance on a granular level. In our example we illustrate this concept by using the
Following Marshal McLuhan’s perspective on media as 'extensions of man', sonification for the ge... more Following Marshal McLuhan’s perspective on media as 'extensions of man', sonification for the generation of knowledge can be regarded as an extension of our auditory sense toward previously imperceptible properties of our environment. Investigating our own involvement from an ontological perspective allows us to gen- erate conceptual handles for the research, development and use of tools for sonification and the implied extension of our physical body through technology. Based on the nature of our bodies as mediators between the shared exterior and the individual interior, a model of three problematic areas of our extended bodies is pre- sented: the cognitive, the physical and the extended.
When we research, design and develop new applications and methods in sonification, we investigate the models and metaphors used in each of these areas, but it is only when we use the devel- oped applications that we actually understand what potentials of perception and exploration we are provided with. It is therefore not sufficient to only build an exterior apparatus: The extended body is each of our own—each researcher and user of sonification develops an individual relationship to all affordances provided
We present an innovative remix-instrument that allows to create music from a collection of audio-... more We present an innovative remix-instrument that allows to create music from a collection of audio-visual media fragments. A three-dimensional scatter plot derived from featureanalysis becomes a Theremin-inspired instrument that enables exploration, intuitive navigation and embodied performance of the media fragments on a granular level. As example, we are using the audiovisual recording of an instrumental performance as source for interactive remix.
The auditory environment has been described as a biased competition : The juxtaposition of an arr... more The auditory environment has been described as a biased competition : The juxtaposition of an array of pre-formed auditory streams and a process of attentional selection . The orientation of attentional selection toward environmental streams is differentiated towards different modes of streaming: Speech, music and sound effects are only three examples in a potentially open polymorphism of perceptual strategies through which we access the sounding world.
… . of the 14 International Conference on …, Jan 1, 2008
Signatures as a sonification strategy that accommodates these demands as well as the underlying c... more Signatures as a sonification strategy that accommodates these demands as well as the underlying concepts of quantitative and qualitative listening are presented. The implementation of these concepts within the interdisciplinary project "ATLAS in silico" is described.
Sensate Abstraction: Hybrid Strategies for Multi-Dimensional Data in Expressive Virtual Reality Contexts
Proc. of ICAD, Jan 1, 2005
In applications of sonification, the information inferred by the sonification strategy applied of... more In applications of sonification, the information inferred by the sonification strategy applied often supersedes the amount of information which can be retrieved by the ear about the object of sonification. This paper focuses on the representation of complex geometric formation through sound, drawing on the development of an interactive installation sonifying escape time fractals as an example. The terms "auditory emergence and formation" are introduced and an attempt is made to interpret them for music composition, data display and information theory. The example application,. "Audio Fraktal", is a public installation in the permanent exhibition of the Museum for Media Art at ZKM, Karlsruhe. The design of the audiovisual display system that allows the shared experience of interactive spatial auditory formation is described.
Proceedings of the 2006 …, Jan 1, 2006
The Klangdom is an audio spatialization instrument developed at the Institut für Musik und Akusti... more The Klangdom is an audio spatialization instrument developed at the Institut für Musik und Akustik at the ZKM. It is made up of 39 Meyer Sound loudspeakers hung on four sliding tracks, allowing for easy re-configuration of the speaker setup. The audience sits inside the Klangdom, which can be controlled either directly via a mixer, by externally developed software, or by a sequencer for sound movement, Zirkonium, developed at the ZKM. Zirkonium can accept and spatialize audio generated by other applications (even on remote machines) and can simulate the Klangdom over alternate speaker setups to aid composition and dissemination (e.g., in stereo or 5.1).
The Spatial Sound Lab at Fraunhofer IMK. VE
… of the 2003 International Conference on …, Jan 1, 2003
Page 1. Proceedings of the 2003 International Conference on Auditory Display, Boston, MA, USA, 6-... more Page 1. Proceedings of the 2003 International Conference on Auditory Display, Boston, MA, USA, 6-9 July 2003 ICAD03-103 THE SPATIAL SOUND LAB AT FRAUNHOFER IMK.VE Joachim Goßmann Dr. Florian Dombois Fraunhofer ...
Networking Virtual Sound and Physical Space in Audio-Augmented Environments
ICAD, Jan 1, 2004
Personal and Ubiquitous Computing, 2002
Music-Pipeline
Tag der offenen Tür| 2006, Jan 1, 2006
Dissertation by Joachim Gossmann
Temporal Metaphorics: Involvement and Participation in Time-Based Media Environments
Uploads
Papers by Joachim Gossmann
When we research, design and develop new applications and methods in sonification, we investigate the models and metaphors used in each of these areas, but it is only when we use the devel- oped applications that we actually understand what potentials of perception and exploration we are provided with. It is therefore not sufficient to only build an exterior apparatus: The extended body is each of our own—each researcher and user of sonification develops an individual relationship to all affordances provided
Dissertation by Joachim Gossmann