Figure 14 – uploaded by Wolfgang Broll

See full PDF downloadDownload figure

Figure 1 effectively and in real time [8]. Even the basic and generic interaction techniques, such as manipulation, coping, annotating, dynamically adding and deleting virtual objects to the MR Figure 1: Tiles environment: users arrange data on the whiteboard, using tangible data containers, data tiles, and adding annotations using whiteboard pen.

Related Figures (34)

To support our initial investigation, we designed and implemented a software framework for application development on the Visual Haptic Workbench. Our software libraries contain haptic rendering methods, general VR support for immersive environment rendering and interaction, visualization methods, interface widgets, dataset classes, menu functions, and geometry tessellators. These software libraries are realized as runtime daemons and application threads, which communicate via shared memory data models and UDP messages. To maintain interactive update rates, these processes run concurrently on an SGI Onyx2 with 250 MHz R1IN0N0N0 nrocesgors and InfiniteR eality? oranhicc

Broll [2] presented some strategies to solve concurrent interaction over a single object. Margey [6] presents a study concerning about the different aspects of cooperative interaction but does not deal with immersive environments. Noma [7] presents a study about cooperative manipulation using force feedback devices.

The two local copies receive the new Command C

Figure 1: Multi-modal interaction: a user speaks and gestures to achieve a desired interaction, in this case the connection of two virtual objects. Work is done on both sides, on the development of multi-modal interpretation and integration (MMI) and on the enhancement of VR-technology and realism. Despite this fact, there are few approaches that deal with the systematic integration of the MMlI-results under general VR-conditions. The latter justifies the foundation for the development of the PrOSA (Patterns On Sequences of Attributes) [3] concepts as fundamental building blocks for multi-modal interaction in VR.

head) and associated reference rays’, line segments that represent deictic or iconic directional and _ orientational information (pointing direction, palm normal, etc.). Actuators deliver a set of their resulting synchronous movement samples for each frame and feed them into higher level processing units like detectors (and their subtypes) or into motion-modificators

Consider a hand-held Augmented Reality Interface (ARI) as illustrated in Fig.l. (Think of a future PalmPilot or cell-phone with a video camera and display.) Imagine that as the user moves this device through the world, information services available in the immediate space are shown on the display. For example, in your office, the news-service node advertises the current market quotes and headlines that match your interests. At the same time, the email portal announces five pending messages. This and other information is available in the space you currently occupy via low power RF or IR connections. You select the email portal by tapping the display or pointing the ARI at the physical device. In the latter case your gesture motions and pointing direction are sensed in the ARI by ceramic or MEMS rate gyroscopes and the video motion field. Your customized email browser appears with the messages and you select the messages of interest by voice input,

Figure 2: Motion fields computed with hybrid algorithm The example above illustrates some of the future possible ARI functionality Rather than

Figure 3: Tracked ARI keeps rack annotations attached to correct features regardless of ARI motion. The raw video sequences were obtained at NASA JSC, Houston system. (Note that Moore's law predicts it will operation information to the user. Note that the

Figure 2: The usability test with the fake PDA city info.

Figure 2: A vision of the future 3D/3G de- vice with GPS. The device has a flexible, in- scrollable display.

Figure 2: The user, wearing lightweight head-mounted display with mounted camera, can see both virtual images registered on tiles and real objects.

virtual objects without using any special purpose

Figure 3: The user cleans data tiles using trash can operator tile. The removed virtual instrument is animated to provide the user with smooth feedback. The current design of MR interfaces, falls into two orthogonal approaches: tangible interfaces based on tabletop MR offer seamless interaction with physical and virtual objects but results in spatial discontinuities. Indeed, the users are unable to use MR environment beyond the limits of the instrumented area, e.g. a projection table. 3D AR _ provides _ spatially seamless MR workspaces, e.g. the user can freely move within the environment, tracked by GPS, magnetic trackers, etc. However, it introduces discontinuities in interaction: different tools has to be used to interact with virtual and physical objects, breaking the natural workflow. In Tiles we attempt to merges the best qualities of interaction styles: true spatial registration of 3D virtual objects anywhere in the space and a tangible interface that allows to interact with

Table 1: Tiles operations: bringing together menu tile and empty data tile moves instrument on the tile (first row).

Figure 1: Virtual (left) and physical (right) overviews of the CMP We believe that tangible instancing of virtual objects with 3D printers, and use of those instances in VR interfaces, is a worthy topic for future research. large board with an array of edge connector slots enabling the connection of numerous objects (each carrying a unique diode-based code) while tracing their location and identification in real- time (see figure 1). Recently, the Segal model was modernized so it can connect to a PC through a standard parallel port, using a Linux driver to scan the board and a Half-life® computer-game-engine to perform the rendering [14].

Table | : Results using MEI To make the implementation simpler, the ime duration for the MEI was substituted in terms of frames. The gesture recognition algorithm was initially first trained by detecting the 3 gestures over an interval of 15 frames each. The 3 gestures were then randomly shown 10 limes each and the results recorded down. Another 2 more sets (B and C) were then included, each with another 3 different random vestures (all in all, a total of 90 instances were tested). To calculate the moments of the MEI, the 3rd and 5th order invariant moments were used. The results using MEI was then shown in Table 1. The results obtained from all 3 sets averaged at 80% success, which is quite acceptable.

Figure 1 : (a-c) the different positions before the person sits down at frame 0 , 5 and 15 (d-f) the MEls at frame 0 , 5 and 15. Note that frame 15 is a summation of frames 0 to 14.

As described above, a gesture and EEG color recognizer was developed in order to provide a natural and seamless interface with the virtual world. To provide a demonstration we developed system which contained a Sony Glasstron head mounted display, computer, and BioSemi EEG machine. The application we developed allowed a user to interact with a virtual environment that contains a virtual vase.

Figure 2: The rendering of the vase with different color. The vase orientation is naturally controlled by the gesture, and the vase’s color is controlled by the user thinking of a color Note that if a color was not recognized a default texture was displayed on the vase surface.

Op-Shop is a virtual room cluttered from floor to ceiling with glasses, vases, bottles, figurines, carpets, and other bric-a-brac. In the middle is a kitschy art-deco table with a champagne glass. You can shatter this glass by singing a high note, just like opera singers in the movies sometimes do. You will be pleased to know that you can ikewise shatter the table - by singing a low note. Op-shop is the perfect place to practice to become an opera star ! Each object responds to acoustic energy based on its size and shape. Weichei is childrens game designed for the Cyberstage Virtual Reality room was shown at the "Virtuelle Welten Erleben" (Discover Virtual Worlds) Art exhibition at the Animax Multimedia Theatre in Bonn <www.animax.de>. When you walk into the room all you see are a dozen football-sized eggs and a giant silver spoon. Each egg hums a little tune and together they produce an a-capella chorus. When you pick up the spoon it whooshes through the air like a sword or light-sabre. You can pick up an egg just like you would with an ordinary spoon, without

BounceMachine is an immersive drum machine for making rhythms and dancing in the same space. There are 20 coloured balls of different sizes on the floor, and a grid floating in the Cyberstage space. If you drop a ball it triggers a sound when it hits the floor. The colours of the balls indicate different percussive instruments - a blue conga, a green cowbell, a purple scratch, an orange snare etc. The size of the ball indicates its heaviness - heavy balls make louder sounds and cause the floor to vibrate when they hit. You can make rhythms by dropping different balls from different heights and letting them bounce. The grid floating in the space helps you position the balls at exact heights to synchronise rhythms. Although its fun it does not really work very well if you want to make tight rhythms. One problem was that it is really very difficult to drop 2 balls from exactly the same height, especially if one of the balls is bouncing. Secondly you don’t hear the ball you drop until it hits the floor which means you have to predict the delay rather than respond to the current beat. Finally the temporal resolution of the physical simulation is still too The Beethoven Salon is a model of the room where Beethoven composed the Pastoral symphony. This model acts as a portal to an abstract world of expression generated by the music. Furnishings and artefacts from the Beethoven Haus collection are seamlessly integrated with matching colours, lighting and materials in the virtual extension. Visible on the virtual half of the piano is the manuscript of the symphony inscribed with the famous phrase "more a feeling than a painting" in Beethoven’s writing. As visitors enter the room a breeze tustles the curtains and turns the pages to the 4th movement - the thunderstorm. As the music begins the lights dim until only glowing silver music notation is visible on the page of the manuscript. Slowly the notes float up off the page leaving silver staves trailing behind like will-o-wisps. During the tranquil opening passages the notes intertwine slowly like a school of fish, leaving gentle symmetrical wakes. As tension builds the notes become agitated, moving faster, further and more abruptly, until the climactic thunder scatters them to all corners of the room. As the movement progresses the notes dive back and forth, coming together in

Larger objects resonate at lower frequencies and more complicated objects are more fragile. As an object accumulates energy it begins to vibrate and feedback a ringing tone. A large urn requires the concentrated attentions of several divas singing together to rupture it. Because the interaction does not require a special tool, divas can join in or leave the choir at any time, and collaborate on equal footing. OpShop demonstrates the notion of using non-verba auditory gestures to interact with objects that are out of arms reach in a human-sized virtua environment. Perhaps the most significant observation of people in OpShop is_ their reluctance to sing in public. I often have to do al the hard work, unless there is someone who can really sing and has an outgoing personality, in which case it can be a lot of fun. The Beethoven Salon is a room-sized Virtual Environment installation designed for the Beethoven Haus Museum in Bonn. The experience is centred on the Pastoral Symphony which has five movements inspired by Beethoven's feelings for nature. Although the symphony contains musical references to nature Beethoven expressly did not want the Pastoral to be taken literally and inscribed the first manuscript with the words "Mehr Ausdruck der ait 66. Empfindung als Malerei"-“‘more an expression of feeling than a painting”.

computer generated virtual environment. They respond to both the music and other sounds in the room. If someone laughs they flitter, and handclaps will startle and scatter them. This responsiveness blurs the line between the real and virtual so the notes seem alive and present, rather than mere computer generated projections in another space. The Beethoven Salon also has acoustic subwoofers under the floor so that the music can be felt through the feet, especially the thunderous sections. This acoustic floor provides the possibility for people who are deaf like Beethoven was to also experience his music. The notes are artificial-life (a-life) algorithms that model the behavior of flocks, schools, and herds of animals. The a-life are specially designed to react to the dynamic tension of Beethoven's music in an expressive way; These algorithms bring an abstract "nature" into the

Figure 1: Facial Communication The present paper assumes the second more general view. A face as an_ independent communication channel.

of Commedia del Arte which are rich in emotions and will be easily understood by the environment participants. Essentially we are driven in the design of the avatars by two aims. Initially to create an avatar that can be perceived as the representation of other participants, and also to facilitate the communication between participants by means of emotional and expressive avatars. Choosing highly realistic avatars would have required the delivery of sophisticated behaviours and complex choreography (e.g. Set of movements to express some emotions). Further to this initial tests have indicated the high level of expectations users have when presented with a realistic avatar.

Figure 14 – uploaded by Wolfgang Broll

Related Figures (34)

Connect with 287M+ leading minds in your field