Chapter 9 Tracking

Advances in Transdisciplinary Engineering, 2021

Virtual Reality (VR) and Augmented Reality (AR) are becoming useful tools to provide a visual representation of a product, a design or environment without the cost of building a prototype. Other applications include maintenance support, procedures development and workstation design. Currently, VR and AR systems can only display visual and audio cues. For example, in workstation design, i.e., a ship’s bridge, aircraft flight deck, command centre, maintenance facility, etc., it is important to for design engineering to determine if the human operator can reach the gauges, switches and controls. To meet the ergonomic and human factor design criteria, the human operator must easily interact with all the displays, instruments, equipment, and controls. This impacts the design and location of seats, instrument panels and windows. VR and AR devices, such as the controllers and gloves, can track the position of the human hand and fingers in a virtual environment but are not able to detect re...

Ethos, 1999

This paper considers the experience of embodiment in current and (possible) future virtual reality applications. A phenomenological perspective is adopted to explore user embodiment in those virtual reality applications that both do and do not include a visual body (re)presentation (virtual body). Embodiment is viewed from the perspective ofsensorial immersion, 'where issues of gender, race, and culture are all implicated. Accounts of "disrupted" bodies (for example, phantom limb and dissociation of the self from the body, paralysis, and objectified bodies) are advanced in order to provide a context for understanding the ways in which embodiment in virtual reality environments may be instantiated. The explicit claim that virtual reality is an embodied experience and can facilitate the radical transfiguration of the body and its sensorial architecture is explored and evaluated.

2021

Inserting a self-representation in Virtual Reality is an open problem with several implications for both the sense of presence and interaction in virtual environments. To cope the problem with low cost devices, we devise a framework to align the measurements of different acquisition devices used while wearing a tracked VR head-mounted display (HMD). Specifically, we use the skeletal tracking features of an RGB-D sensor (Intel Realsense d435) to build the user’s avatar, and compare different interaction technologies: a Leap Motion, the Manus Prime haptic gloves, and the Oculus Controllers. The effectiveness of the proposed systems is assessed through an experimental session, where an assembly task is proposed with the three different interaction mediums, with and without the self-representation. Users reported their feeling by answering the User Experience and Igroup Presence Questionnaires, and we analyze the total time to completion and the error rate.

2000

In this paper, we describe a user interface technique that uses a bounding box as a m taphor to facilitate interaction in a Virtual Reality (VR) environment. Because this technique is based on the observation that some of the VR application fields are contained in a closed world, we call it Closed World Interaction (CWI). After the user defines a closed world, the necessar virtual buttons are shown around the closed world which is presented by a fi'ame. These virtual buttons are then used to interact with models. We also integrate some of the 2D Windows, Icons, Mouse and Pointer (WIMP) metaphors into CWI technique, reflecting our belief that users will be able to adapt to this environment quickly, A series of user studies were conducted to investigate the effectiveness of this technique. The results indicate that users can define a closed world quickly. Experience appears to be an important factor, and users can be trained to become familiar with CWI in the VR environntent. The constrained interactions can also enhance the accuracy of selection. Two-handed manipulation somewhat improves the speed.

Proc. SVR

When interacting in the real world, it is common to remember locations of objects based on our own body previous locations and postures. In this paper we discuss the benefits of the whole-body awareness in 3d interactive applications. We propose a technique for navigation and selection in 3d environments which uses an optical tracking system to provide whole-body position and orientation. We explore the peephole metaphor with a tablet pc to artificially enlarge the display and interaction area. In the application implemented to prove the proposed concepts, the tablet is held by the participant who moves it around and points it in any direction for visualization and interaction.

Proceedings of the …, 2010

2019

This paper presents a novel interaction technique to combine multiple inputs and output modalities with a 3D printed open disc which is tracked and augmented with a virtual information. Props have been used in virtual and augmented environments to provide tangible associations to virtual elements. When used in a larger immersive environment or where a user is walking around, users may not have the capability to set these objects/props down or are limited to the number of hands, so users are reduced to the capability of using only two props or controllers. In our technique, users can instead use their body to store and organize the virtual objects associated with tangible props by physically hanging them on or attaching them to the body. This type of interaction is well-suited for both immersive visualizations and immersive virtual environments. In this paper, we present the results of an experimental evaluation of our Body Prop Interaction technique for each type of immersive environment and corresponding task type, demonstrating more effective usage in each scenario than a joystick or gesture-based interaction.

Virtual Reality and Its Application in Education, 2021

2020

In recent years head-mounted displays (HMDs) for virtual reality (VR) have made the transition from research to consumer product, and are increasingly used for productive purposes such as 3D modeling in the automotive industry and teleconferencing. VR allows users to create and experience real-world like models of products; and enables users to have an immersive social interaction with distant colleagues. These solutions are a promising alternative to physical prototypes and meetings, as they require less investment in time and material. VR uses our visual dominance to deliver these experiences, making users believe that they are in another reality. However, while their mind is present in VR their body is in the physical reality. From the user's perspective, this brings considerable uncertainty to the interaction. Currently, they are forced to take off their HMD in order to, for example, see who is observing them and to understand whether their physical integrity is at risk. This disrupts their interaction in VR, leading to a loss of presence-a main quality measure for the success of VR experiences. In this thesis, I address this uncertainty by developing interfaces that enable users to stay in VR while supporting their awareness of the physical reality. They maintain this awareness without having to take off the headset-which I refer to as seamless interaction with the physical reality. The overarching research vision that guides this thesis is, therefore, to reduce this disconnect between the virtual and physical reality. My research is motivated by a preliminary exploration of user uncertainty towards using VR in co-located, public places. This exploration revealed three main foci: (a) security and privacy, (b) communication with physical collaborators, and (c) managing presence in both the physical and virtual reality. Each theme represents a section in my dissertation, in which I identify central challenges and give directions towards overcoming them as have emerged from the work presented here. First, I investigate security and privacy in co-located situations by revealing to what extent bystanders are able to observe general tasks. In this context, I explicitly investigate the security considerations of authentication mechanisms. I review how existing authentication mechanisms can be transferred to VR and present novel approaches that are more usable and secure than existing solutions from prior work. Second, to support communication between VR users and physical collaborators, I add to the field design implications for VR interactions that enable observers to choose opportune moments to interrupt HMD users. Moreover, I contribute methods for displaying interruptions in VR and discuss their effect on presence and performance. I also found that different virtual presentations of co-located collaborators have an effect on social presence, performance and trust. iii v Drittens, schließe ich meine Dissertation mit der Untersuchung von Methoden zur Verwaltung der Präsenz, sowohl in der physischen als auch in der virtuellen Realität ab. Ich schlage Systeme und Schnittstellen für den Übergang zwischen den Realitäten vor, die die Benutzer in die Lage versetzen zu entscheiden, inwieweit sie sich der anderen Realität bewusst sein wollen. Schließlich diskutiere ich die Möglichkeit, diesen beiden Realitäten systematisch Sinne zuzuordnen: die visuelle für VR und die auditive und haptische für die physische Realität. Darüber hinaus stelle ich spezifische Design-Richtlinien zur Verfügung, wie diese Erkenntnisse genutzt werden können, um VR-Anwender auf physische Grenzen und Hindernisse aufmerksam zu machen. vi ACKNOWLEDGMENTS First and foremost, I would like to thank Heinrich Hussmann, for providing a very flexible work environment that allowed me to pursue my PhD while raising my children. I appreciate the collaborative & trustworthy environment that you created together with Andreas Butz that makes working a joy. You made it easy to think of colleagues as friends. I would like to thank Sid Fels for the inspiring and encouraging conversations and for always being so positive. Katrin Wolf, thank you for reviewing my thesis. I am glad our paths have crossed. Further, I would like to thank the amazing people at my lab, starting with the people I got to know first. Hanna Schneider, thank you for the wise words when I was just starting off and for lots and lots of laughter-they really made my day. Renate Häuslschmid, thank you for rationalizing with me when I was going crazy about not being able to do things. Malin Eiband for making me feel stupid. You have been a strong support from day one. Sarah Völkel thank you for the long not-work-related chats at work. Every day at work without you is a sad one. Introduction Today the term virtual reality is applied to a number of concepts, often overlapping with the words world and environment, conceptually as well as in nomenclature. This is also prominent in my publications, as each paper focuses on an individual subset of prior work, where one term may be favored over the other. 1.3 Usable Security and Privacy for VR: Observability of HMD User Interactions "ALEX is in a completely empty room without anything and anyone" (Participant 26) Preliminary Exploration [33] Privacy was found to be an unvoiced concern in our preliminary exploration [P1]. As part of the story completion method, the majority of participants placed their fictional character ALEX alone in a room, specifically mentioning the door being locked. We found that they were worried about co-located people and being observed by them. However, observability through bystanders was not exclusively mentioned in a negative context. The relationship to the observer paid a role in determining whether it was positive or negative. A close Hence, the first goal of this thesis is to understand what interactions/gestures of an HMD user can be observed by a bystander. RQ1 A: Which tasks of an HMD user can be observed by a bystander? Eiband et al. found that 9% of attacks were on users' credentials [15]. Stealing login details through shoulder surfing is predominantly investigated on ubiquitous devices, as users are able to use them in public places with unknown bystanders. On mobile phones results highlighted that shoulder surfing risk is perceived to be low, independent of how high it actually is [39]. Thus, the risk has to mitigated by the system rather than relying on the user to take action. RQ1 B: How to create and test authentication mechanisms for VR users that are both usable and secure? Contribution. To explore RQ1a, this thesis presents a user study that investigated which tasks and task switches can be observed by bystanders [P5]. Tasks were based on common tasks that are currently completed on ubiquitous devices, as derived from a preliminary diary study we completed [P5] where participants had to keep a diary about their digital media usage and respond to pre-defined questions. Based on these results we included five tasks in our study: Watching a video, typing a text, authenticating, reading a text and 3D manipulation. Eiband et al. [15] found a similar set of tasks as a result of their shoulder surfing risk study on mobile phones. Instead of 3D manipulation their list included gaming, which we excluded, as our setup/story was an office environment with architects. We found bystanders were able to successfully identify both task switches (83%) and tasks (77%) within only a few seconds of the task switch. We also provide design recommendations (e.g. pointer vs tapping to increase security) for implementing less observable VR interactions and suggest concepts that enable privacy in co-located work spaces with HMDs. 9 Contribution. To investigate RQ2a our approach was twofold: First, we explored how bystanders interrupt an HMD user and whether they are capable of observing the HMD user's task switches, solely through interpreting their gestures [P5]. We found that bystanders tended to use voice interruptions during task completion and that repetition improved interruption behavior, thus showing the capability of interrupting in opportune moments upon training. Based on these results, we propose interaction concepts that enable bystanders to choose opportune moments for interruption, for example, the need for prominent vs. subtle gestures. We also propose concepts that include the layout of the physical reality, whereby the user is guided to stand facing a specific direction. For example, the user could be positioned to face the bystander to indicate openness for interruption, or with their back to the bystander to show that they do not want to be disturbed. This does not affect their VR expe-sistance of 3D Passwords in virtual Reality

Frontiers in Robotics and AI: Virtual Environments, 2019

It is currently not fully understood where people precisely locate themselves in their bodies, particularly in virtual reality. To investigate this, we asked participants to point directly at themselves and to several of their body parts with a virtual pointer, in two virtual reality (VR) setups, a VR headset and a large-screen immersive display (LSID). There was a difference in distance error in pointing to body parts depending on VR setup. Participants pointed relatively accurately to many of their body parts (i.e., eyes, nose, chin, shoulders, and waist). However, in both VR setups when pointing to the feet and the knees they pointed too low, and for the top of the head too high (to larger extents in the VR headset). Taking these distortions into account, the locations found for pointing to self were considered in terms of perceived bodies, based on where the participants had pointed to their body parts in the two VR setups. Pointing to self in terms of the perceived body was mostly to the face, the upper followed by the lower, as well as some to the torso regions. There was no significant overall effect of VR condition for pointing to self in terms of the perceived body (but there was a significant effect of VR if only the physical body (as measured) was considered). In a paper-and-pencil task outside of VR, performed by pointing on a picture of a simple body outline (body template task), participants pointed most to the upper torso. Possible explanations for the differences between pointing to self in the VR setups and the body template task are discussed. The main finding of this study is that the VR setup influences where people point to their body parts, but not to themselves, when perceived and not physical body parts are considered.