Interface with angels: the future of VIR and AR interfaces

GMD–Forschungszentrum Informationstechnik GmbH, 2001

What can Virtual Reality and Augmented Reality technologies do for us? Can they help us make better use of computers? Will they ever improve the quality of our personal lives? What kind of immersive interfaces could re-shape the way we communicate with computers? How far along are we on that road, and what remains to be done? This is only a small selection of the questions discussed at the workshop on the future of VR and AR interfaces at IEEE Virtual Reality 2001. Main areas of focus were conversational user interfaces, natural ...

Computer Graphics and …, 2001

Encyclopedia of Multimedia, 2006

Virtual and augmented reality technologies have entered a new near-commodity era, accompanied by massive commercial investments, but still are subject to numerous open research questions. This special issue of IEEE Computer Graphics and Applications aims at broad views to capture the state of the art, important achievements, and impact of several areas in these dynamic disciplines. It contains three original articles that consider important aspects of VR/AR technologies and outline future research opportunities. These are special times for virtual reality (VR) and augmented reality (AR). These technologies have just entered the life and the imagination of general audiences in a spectacular way. Only two years ago, hardly anybody-apart from some specialists and geeks-had experienced headmounted displays, and such headsets were still considered futuristic and expensive gadgets, limited to research or purpose-built application use. But today, anybody can purchase a complete VR setup at various consumer electronics price points at a neighborhood or online electronics retailer. These devices come complete with sophisticated and effective display and tracking technologies, and showcase impressive 3D contents and experiences, realistic and artistic. AR is following suit, with great promise and technologically advanced prototypes from major industrial players that are heavily invested in the technology.

IEEE Computer Graphics and Applications, 2001

2020

In recent years head-mounted displays (HMDs) for virtual reality (VR) have made the transition from research to consumer product, and are increasingly used for productive purposes such as 3D modeling in the automotive industry and teleconferencing. VR allows users to create and experience real-world like models of products; and enables users to have an immersive social interaction with distant colleagues. These solutions are a promising alternative to physical prototypes and meetings, as they require less investment in time and material. VR uses our visual dominance to deliver these experiences, making users believe that they are in another reality. However, while their mind is present in VR their body is in the physical reality. From the user's perspective, this brings considerable uncertainty to the interaction. Currently, they are forced to take off their HMD in order to, for example, see who is observing them and to understand whether their physical integrity is at risk. This disrupts their interaction in VR, leading to a loss of presence-a main quality measure for the success of VR experiences. In this thesis, I address this uncertainty by developing interfaces that enable users to stay in VR while supporting their awareness of the physical reality. They maintain this awareness without having to take off the headset-which I refer to as seamless interaction with the physical reality. The overarching research vision that guides this thesis is, therefore, to reduce this disconnect between the virtual and physical reality. My research is motivated by a preliminary exploration of user uncertainty towards using VR in co-located, public places. This exploration revealed three main foci: (a) security and privacy, (b) communication with physical collaborators, and (c) managing presence in both the physical and virtual reality. Each theme represents a section in my dissertation, in which I identify central challenges and give directions towards overcoming them as have emerged from the work presented here. First, I investigate security and privacy in co-located situations by revealing to what extent bystanders are able to observe general tasks. In this context, I explicitly investigate the security considerations of authentication mechanisms. I review how existing authentication mechanisms can be transferred to VR and present novel approaches that are more usable and secure than existing solutions from prior work. Second, to support communication between VR users and physical collaborators, I add to the field design implications for VR interactions that enable observers to choose opportune moments to interrupt HMD users. Moreover, I contribute methods for displaying interruptions in VR and discuss their effect on presence and performance. I also found that different virtual presentations of co-located collaborators have an effect on social presence, performance and trust. iii v Drittens, schließe ich meine Dissertation mit der Untersuchung von Methoden zur Verwaltung der Präsenz, sowohl in der physischen als auch in der virtuellen Realität ab. Ich schlage Systeme und Schnittstellen für den Übergang zwischen den Realitäten vor, die die Benutzer in die Lage versetzen zu entscheiden, inwieweit sie sich der anderen Realität bewusst sein wollen. Schließlich diskutiere ich die Möglichkeit, diesen beiden Realitäten systematisch Sinne zuzuordnen: die visuelle für VR und die auditive und haptische für die physische Realität. Darüber hinaus stelle ich spezifische Design-Richtlinien zur Verfügung, wie diese Erkenntnisse genutzt werden können, um VR-Anwender auf physische Grenzen und Hindernisse aufmerksam zu machen. vi ACKNOWLEDGMENTS First and foremost, I would like to thank Heinrich Hussmann, for providing a very flexible work environment that allowed me to pursue my PhD while raising my children. I appreciate the collaborative & trustworthy environment that you created together with Andreas Butz that makes working a joy. You made it easy to think of colleagues as friends. I would like to thank Sid Fels for the inspiring and encouraging conversations and for always being so positive. Katrin Wolf, thank you for reviewing my thesis. I am glad our paths have crossed. Further, I would like to thank the amazing people at my lab, starting with the people I got to know first. Hanna Schneider, thank you for the wise words when I was just starting off and for lots and lots of laughter-they really made my day. Renate Häuslschmid, thank you for rationalizing with me when I was going crazy about not being able to do things. Malin Eiband for making me feel stupid. You have been a strong support from day one. Sarah Völkel thank you for the long not-work-related chats at work. Every day at work without you is a sad one. Introduction Today the term virtual reality is applied to a number of concepts, often overlapping with the words world and environment, conceptually as well as in nomenclature. This is also prominent in my publications, as each paper focuses on an individual subset of prior work, where one term may be favored over the other. 1.3 Usable Security and Privacy for VR: Observability of HMD User Interactions "ALEX is in a completely empty room without anything and anyone" (Participant 26) Preliminary Exploration [33] Privacy was found to be an unvoiced concern in our preliminary exploration [P1]. As part of the story completion method, the majority of participants placed their fictional character ALEX alone in a room, specifically mentioning the door being locked. We found that they were worried about co-located people and being observed by them. However, observability through bystanders was not exclusively mentioned in a negative context. The relationship to the observer paid a role in determining whether it was positive or negative. A close Hence, the first goal of this thesis is to understand what interactions/gestures of an HMD user can be observed by a bystander. RQ1 A: Which tasks of an HMD user can be observed by a bystander? Eiband et al. found that 9% of attacks were on users' credentials [15]. Stealing login details through shoulder surfing is predominantly investigated on ubiquitous devices, as users are able to use them in public places with unknown bystanders. On mobile phones results highlighted that shoulder surfing risk is perceived to be low, independent of how high it actually is [39]. Thus, the risk has to mitigated by the system rather than relying on the user to take action. RQ1 B: How to create and test authentication mechanisms for VR users that are both usable and secure? Contribution. To explore RQ1a, this thesis presents a user study that investigated which tasks and task switches can be observed by bystanders [P5]. Tasks were based on common tasks that are currently completed on ubiquitous devices, as derived from a preliminary diary study we completed [P5] where participants had to keep a diary about their digital media usage and respond to pre-defined questions. Based on these results we included five tasks in our study: Watching a video, typing a text, authenticating, reading a text and 3D manipulation. Eiband et al. [15] found a similar set of tasks as a result of their shoulder surfing risk study on mobile phones. Instead of 3D manipulation their list included gaming, which we excluded, as our setup/story was an office environment with architects. We found bystanders were able to successfully identify both task switches (83%) and tasks (77%) within only a few seconds of the task switch. We also provide design recommendations (e.g. pointer vs tapping to increase security) for implementing less observable VR interactions and suggest concepts that enable privacy in co-located work spaces with HMDs. 9 Contribution. To investigate RQ2a our approach was twofold: First, we explored how bystanders interrupt an HMD user and whether they are capable of observing the HMD user's task switches, solely through interpreting their gestures [P5]. We found that bystanders tended to use voice interruptions during task completion and that repetition improved interruption behavior, thus showing the capability of interrupting in opportune moments upon training. Based on these results, we propose interaction concepts that enable bystanders to choose opportune moments for interruption, for example, the need for prominent vs. subtle gestures. We also propose concepts that include the layout of the physical reality, whereby the user is guided to stand facing a specific direction. For example, the user could be positioned to face the bystander to indicate openness for interruption, or with their back to the bystander to show that they do not want to be disturbed. This does not affect their VR expe-sistance of 3D Passwords in virtual Reality