Haptic Control of Multistate Generative Music Systems

2002

This paper describes a new computer-based music synthesiser that is based on physical modelling. This instrument operates in real-time and provides the player with gestural control and tactile as well as acoustic feedback. The physical modelling makes use of a mass/spring paradigm, and multi-dimensional, interconnectable resonating structures can be created, and these can be excited with a variety of methods. Gestural control is provided via a force feedback joystick and a tactile feedback mouse. The use of gestural control assures that the output is subtly different on each occasion. The acoustic output from the system is perceived as being highly natural and organic.

This chapter introduces to the concept of musical haptics, its scope, aims, challenges, as well as its relevance and impact for general haptics and human– computer interaction. A brief summary of subsequent chapters is given.

2005

PHASE is a research project concerning multi-modal systems for generation, handling and control of sound and music. Some of the research results have been implemented in an interactive installation, a musical game integrating various metaphors. This installation, open to the public at the G. Pompidou museum for three months, was an extraordinary success showing the validity of such a device and revealing many possibilities of gestural music control.

Proceedings of the …, 2002

The trend towards the virtual in music synthesis has lead to the inevitable decline of the physical, effectively inserting a 'veil of tactile paralysis' between the musician and the sound source. Cymatic is a new computer based musical instrument which attempts to reverse ...

Despite enjoying the resulting freedom from physical constraints that can result from the incorporation of computer technology in electronic musical instruments as musicians are enabled to push back the creative boundaries within which they work, musicians are still searching for virtual instruments that come closer to their physical counterparts in terms of their playing experience. This paper describes a physical modelling music synthesis system known that enables 'virtual instruments' to be controlled in real-time via a force-feedback joystick and a force-feedback mouse. These serve to provide the user with gestural controllers incorporating tactile feedback. Virtual instruments are set up via a graphical user interface in a manner that is highly intuitive, since users design these virtual instruments by interacting directly with their physical shape and structure in terms of the physical properties of basic objects such as strings, membranes and solids which can be interconnected to form complex structures. Acoustic excitation can be applied at any point mass via virtual bowing, plucking, striking, or from an external sound source. Virtual microphones can be placed at any point masses to deliver the acoustic output. The organic nature of the resulting acoustic output is illustrated.

PhD Thesis, 2014

As a performer of firstly acoustic and latterly electronic and electro-instrumental music, I constantly seek to improve my mode of interaction with the digital realm: that is, to achieve a high level of sensitivity and expression. This thesis illustrates reasons why making use of haptic interfaces—which offer physical feedback and resistance to the performer—may be viewed as an important approach in addressing the shortcomings of some the standard systems used to mediate the performer’s engagement with various sorts of digital musical information. By examining the links between sound and touch, and the performer-instrument relationship, various new compositional and performance strategies start to emerge. I explore these through a portfolio of original musical works, which span the continuum of composition and improvisation, largely based around performance paradigms for piano and live electronics. I implement new haptic technologies, using vibrotactile feedback and resistant interfaces, as well as exploring more metaphorical connections between sound and touch. I demonstrate the impact that the research brings to the creative musical outcomes, along with the implications that these techniques have on the wider field of live electronic musical performance.

2000

In this paper we outline the fundamentals for a tactile feedback system to be used in conjunction with openair computer music performance devices. Some underlying physiological and perceptual mechanisms of haptics are examined, some currently available open-air controllers are reviewed, and previous technologies and experiments regarding haptic/tactile feedback are surveyed. Our VR/TX system is proposed as a solution for adding tactile feedback to open-air controllers; experiments show that the VR/TX vibrotactile stimulators provide invaluable perceptually-significant tactile feedback when used in conjunction with an open-air music controller. A typology of tactile sound events is also described, as well as the notion of a tactile simulation event (TSE).

2004

Electronic sound synthesis continues to offer huge potential possibilities for the creation of new musical instruments. The traditional approach is, however, seriously limited in that it incorporates only auditory feedback and it will typically make use of a sound synthesis model (e.g., additive, subtractive, wavetable, and sampling) that is inherently limited and very often nonintuitive to the musician. In a direct attempt to challenge these issues, this paper describes a system that provides tactile as well as acoustic feedback, with real-time synthesis that invokes a more intuitive response from players since it is based upon mass-spring physical modelling. Virtual instruments are set up via a graphical user interface in terms of the physical properties of basic wellunderstood sounding objects such as strings, membranes, and solids. These can be interconnected to form complex integrated structures. Acoustic excitation can be applied at any point mass via virtual bowing, plucking,...

2005

Sound Navigation consists in browsing through different sound objects and sound generators situated within a virtual world including virtual spatialized sound and visual scenes, to perform a musical trajectory and composition. In the PHASE Project installation, the 3D virtual world resembles the surface of a vinyl disk, magnified so that one can see the groove and move a “needle” (the “reading head”) in it and out of it to read the disk. Another such “needle” (the “writing head”) can “write” music in the groove. A part of the game is a pursuit between the writing head and the reading head handled by the player. Different musical devices have been implemented. Most of them have a haptic behavior. The scenario is fully related to the musical metaphor and aims to give equivalent pleasure to contemplative as well as to competitive players.

New Interfaces for Musical Expression, 2005

The PHASE project is a research project devoted to the study and the realization of systems of multi-modal interaction for generation, handling and control of sound and music. Supported by the network RIAM (Recherche et Innovation en Audiovisuel et Multimédia), it was carried out by the CEA-LIST for haptic research, Haption for the realization of the haptic device, Ondim for integration and visual realization and Ircam for research and realization about sound, music and the metaphors for interaction. The integration of the three modalities offers completely innovative capacities for interaction. The objectives are scientific, cultural and educational. Finally, an additional objective was to test such a prototype system, including its haptic arm, in real conditions for general public and over a long duration in order to measure its solidity, its reliability and its interest for users. Thus, during the last three months of the project, a demonstrator was presented and evaluated in a museum in Paris, in the form of an interactive installation offering the public a musical game. Different from a video game, the aim is not to animate the pixels on the screen but to play music and to incite musical awareness.