Using Mixed Reality Agents as Social Interfaces for Robots

Intelligent Systems, …, 2002

2001

Abstract Interactive virtual worlds provide a powerful medium for entertainment and experiential learning. Our goal is to enrich such virtual worlds with virtual humansautonomous agents that support face-to-face interaction with people in these environments in a variety of roles. While supporting face-to-face interaction in virtual worlds is a daunting task, this paper argues that the key building blocks are already in place.

i-com, 2015

Driven by large industry investments, developments of Virtual Reality (VR) technologies including unobtrusive sensors, actuators and novel display devices are rapidly progressing. Realism and interactivity have been postulated as crucial aspects of immersive VR since the naissance of the concept. However, today’s VR still falls short from creating real life-like experiences in many regards. This holds particularly true when introducing the “social dimension” into the virtual worlds. Apparently, creating convincing virtual selves and virtual others and conveying meaningful and appropriate social behavior still is an open challenge for future VR. This challenge implies both, technical aspects, such as the real-time capacities of the systems, but also psychological aspects, such as the dynamics of human communication. Our knowledge of VR systems is still fragmented with regard to social cognition, although the social dimension is crucial when aiming at autonomous agents with a certain ...

2020

Due to the COVID-19 pandemic, conducting Human-Robot Interaction (HRI) studies in person is not permissible due to social distancing practices to limit the spread of the virus. Therefore, a virtual reality (VR) simulation with a virtual robot may offer an alternative to real-life HRI studies. Like a real intelligent robot, a virtual robot can utilize the same advanced algorithms to behave autonomously. This paper introduces HAVEN (HRI-based Augmentation in a Virtual robot Environment using uNity), a VR simulation that enables users to interact with a virtual robot. The goal of this system design is to enable researchers to conduct HRI Augmented Reality studies using a virtual robot without being in a real environment. This framework also introduces two common HRI experiment designs: a hallway passing scenario and human-robot team object retrieval scenario. Both reflect HAVEN's potential as a tool for future AR-based HRI studies.

Advances in Human-Computer Interaction, 2012

This paper presents the design, development, and testing of a tabletop interface called RoboTable, which is an infrastructure supporting intuitive interaction with both mobile robots and virtual components in a mixed-reality environment. With a flexible software toolkit and specifically developed robots, the platform enables various modes of interaction with mobile robots. Using this platform, prototype applications are developed for two different application domains:RoboPonginvestigates the efficiency of the RoboTable system in game applications, andExploreRobotexplores the possibility of using robots and intuitive interaction to enhance learning.

As robots become more and more embedded in our physical and social environment, their integration into our social interaction space necessitates mechanisms which manage these new social contexts. While considerable work has been invested in developing strong human-like robots in order to arguably augment the human-robot interaction experience, the core complexity and significant costs with such an approach render it difficult to justify for practical real-world applications. This paper discusses the use of augmented reality as a tool to bypass this issue and allow the designer, and subsequent user, to easily choose an aesthetic with associated social behavioural mechanisms. Not only does this allow a user to perceive an associated form with a robot, but also allows many people to perceive alternate forms for the same robot, a degree of social customisation which has been impossible until now.

Intelligent Virtual Agents, 2013

While virtual humans are proven tools for training, education and research, they are far from realizing their full potential. Advances are needed in individual capabilities, such as character animation and speech synthesis, but perhaps more importantly, fundamental questions remain as to how best to integrate these capabilities into a single framework that allows us to efficiently create characters that can engage users in meaningful and realistic social interactions. This integration requires in-depth, inter-disciplinary understanding few individuals, or even teams of individuals, possess. We help address this challenge by introducing the ICT Virtual Human Toolkit 1 , which offers a flexible framework for exploring a variety of different types of virtual human systems, from virtual listeners and question-answering characters to virtual role-players. We show that due to its modularity, the Toolkit allows researchers to mix and match provided capabilities with their own, lowering the barrier of entry to this multidisciplinary research challenge.

Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment

This paper describes work towards the goal of enabling unscripted interaction with non-player characters in virtual environments. We hypothesize that we can define a layer of social affordances, based on physical and non-verbal signals exchanged between individuals and groups, which can be reused across games. We have implemented a first version of that substrate that employs whole body interaction with virtual characters and generates nuanced, real-time character performance in response. We describe the playable experience produced by the system, the implementation architecture (based on the behavior specification technology used in Façade, the social model employed in Prom Week, and gesture recognition technology), and illustrate the key behaviors and programming idioms that enable character performance. These idioms include orthogonal coding of attitudes and activities, use of relational rules to nominate social behavior, use of volition rules to rank options, and priority ba...

Interacting with Computers, 2010

In this paper an interactive and realistic virtual head oriented to human-computer interaction and social robotics is presented. It has been designed following a hybrid approach, taking robotic characteristics into account and searching for a convergence between these characteristics, real facial actions and animation techniques. An initial head model is first obtained from a real person using a laser scanner. Then the model is animated using a hierarchical skeleton based procedure. The proposed rig structure is close to real facial muscular anatomy and its behaviour follows the Facial Action Coding System. Speech synthesis and visual human-face tracking capabilities are also integrated for providing the head with further interaction ability. Using the said hybrid approach, the head can be readily linked to a social robot architecture. The opinions of a number of persons interacting with this social avatar have been evaluated and are reported in the paper, as against their reactions when interacting with a social robot with a mechatronic face. Results show the suitability of the avatar for on-screen, real-time interfacing in human-computer interaction. The proposed technique could also be helpful in the future for designing and parameterizing mechatronic humanlike heads for social robots.

2009

Abstract: This paper discusses the sensorial perceptions of students through interactions with virtual characters and robots converged into mixed reality games, with primary focus on the following topics: whether the robotic collaboration generates greater motivation in learning, students' preferences between interface with virtual characters versus robots as well as the differences in interactivity.