A mixed reality 3D conferencing application

2013 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), 2013

This paper describes an AR system for remote collaboration using a captured 3D model of the local user's scene. In the system a remote user can manipulate the scene independently of the view of the local user and add AR annotations that appear projected into the real world. Results from a pilot study and the design of a further full study are presented.

IEEE Transactions on Circuits and Systems for Video Technology, 2017

This paper presents a mixed reality (MR) system that results from the integration of a telepresence system and an application to improve collaborative space exploration. The system combines free viewpoint video with immersive projection technology to support nonverbal communication (NVC), including eye gaze, interpersonal distance, and facial expression. Importantly, these features can be interpreted together as people move around the simulation, maintaining a natural social distance. The application is a simulation of Mars, within which the collaborators must come to agreement over; for example, where the Rover should land and go. The first contribution is the creation of an MR system supporting contextualization of NVC. Two technological contributions are prototyping a technique to subtract a person from a background that may contain physical objects and/or moving images and a lightweight texturing method for multiview rendering, which provides balance in terms of visual and temporal quality. A practical contribution is the demonstration of pragmatic approaches to sharing space between display systems of distinct levels of immersion. A research tool contribution is a system that allows comparison of conventional authored and video-based reconstructed avatars, within an environment that encourages exploration and social interaction. Aspects of system quality, including the communication of facial expression and end-to-end latency are reported.

BT Technology Journal, 2002

Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, 2019

Remote Collaboration using Virtual Reality (VR) and Augmented Reality (AR) has recently become a popular way for people from different places to work together. Local workers can collaborate with remote helpers by sharing 360-degree live video or 3D virtual reconstruction of their surroundings. However, each of these techniques has benefits and drawbacks. In this paper we explore mixing 360 video and 3D reconstruction together for remote collaboration, by preserving benefits of both systems while reducing drawbacks of each. We developed a hybrid prototype and conducted user study to compare benefits and problems of using 360 or 3D alone to clarify the needs for mixing the two, and also to evaluate the prototype system. We found participants performed significantly better on collaborative search tasks in 360 and felt higher social presence, yet 3D also showed potential to complement. Participant feedback collected after trying our hybrid system provided directions for improvement. • Human-centered computing → Mixed / augmented reality; Collaborative interaction; Computer supported cooperative work.

1998

In this paper we address an application of computer vision which can in the future change completely our way of communicating over the network. We present our version of a testbed for telecollaboration. It is based on a highly accurate and precise stereo algorithm. The results demonstrate the live (on-line) recovery of 3D models of a dynamically changing environment and the simultaneous display and manipulation of the models

2000

Abstract In the Shared Space project, we explore, innovate, de sign and evaluate future computing environments that will radically enhance interaction between human and computers as well as interaction between humans me diated by computers. In particular, we investigate how augmented reality enhanced by physical and spatial 3D user interfaces can be used to develop effective face to face and remote collaborative computing environments. How will we interact in such collaborative spaces?

Citeseer

Some applications require a person in a remote location (the trainer), to instruct another person (the trainee) on how to put together a number of objects according to a precise set of spatial relationships. In this paper we report the results of using a collaborative Mixed Reality application that allows the trainer to manipulate the virtual objects in a virtual environment. At the same time, the trainee observes the virtual object manipulation as means to move the real objects accordingly. The trainer uses a Virtual Reality interface and the trainee uses an Augmented Reality one. The results show that, even though the time to complete the task using the application is longer, users perceive that collaboration using a similar application in the real world could make collaboration easier.

IET 5th European Conference on Visual Media Production (CVMP 2008), 2008

Traditional set-top camera video-conferencing systems still fail to meet the 'telepresence challenge' of providing a viable alternative for physical business travel, which is nowadays characterized by unacceptable delays, costs, inconvenience, and an increasingly large ecological footprint. Even recent high-end commercial solutions, while partially removing some of these traditional shortcomings, still present the problems of not scaling easily, expensive implementations, not utilizing 3D life-sized representations of the remote participants and addressing only eye contact and gesturebased interactions in very limited ways. The European FP7 project 3DPresence will develop a multi-party, high-end 3D videoconferencing concept that will tackle the problem of transmitting the feeling of physical presence in real-time to multiple remote locations in a transparent and natural way. In this paper, we present an overall concept, which includes the geometrical design of the whole prototype demonstrator, the arrangement of the cameras and displays and the general multi-view video analysis chain. The driving force behind the design strategy is to fulfil the requirements of a novel 3D immersive videoconferencing system, including directional eye gaze and gesture awareness.

2004

This paper describes an Augmented Reality Videoconferencing System, which is a novel remote collaboration tool combining a desktop-based AR system and a videoconference module. The novelty of our system is the combination of these tools with AR applications superimposed on live video background displaying the conference parties' real environment, merging the advantages of the natural face-to-face communication of videoconferencing and AR's interaction capabilities with distributed virtual objects using tangible physical artifacts. The simplicity of the system makes it affordable for everyday use. We explain our system design based on concurrent video streaming, optical tracking and 3D application sharing, and provide experimental proof that it yields superior quality compared to pure video streaming with successive optical tracking from the compressed streams. We demonstrate the system's collaborative features with a volume rendering application that allows users to display and examine volumetric data simultaneously and to highlight or explore slices of the volume by manipulating an optical marker as a cutting plane interaction device.

1993

in which participants at diflerent sites can feel as if they are at one site and can cooperatively work. In the teleconferencing system, the 3 0 model of a participant is constructed b y a wire frame model mapped b y color texture and is displayed on the 3 0 screen at the receiving site.