Low-Cost Telepresence for Collaborative Virtual Environments

1993

Several common systems satisfy some but not all of the VR definition above. Flight simulators provide vehicle tracking, not head tracking, and do not generally operate in binocular stereo. Omnimax theaters give a large angle of view [8], occasionally in stereo, but are not interactive. Head-tracked monitors [4][6] provide all but a large angle of view. Head-mounted displays (HMD) [13] and BOOMs [9] use motion of the actual display screens to achieve VR by our definition. Correct projection of the imagery on large screens can also create a VR experience, this being the subject of this paper. This paper describes the CAVE (CAVE Automatic Virtual Environment) virtual reality/scientific visualization system in detail and demonstrates that projection technology applied to virtual-reality goals achieves a system that matches the quality of workstation screens in terms of resolution, color, and flicker-free stereo. In addition, this format helps reduce the effect of common tracking and system latency errors. The off-axis perspective projection techniques we use are shown to be simple and straightforward. Our techniques for doing multi-screen stereo vision are enumerated, and design barriers, past and current, are described. Advantages and disadvantages of the projection paradigm are discussed, with an analysis of the effect of tracking noise and delay on the user. Successive refinement, a necessary tool for scientific visualization, is developed in the virtual reality context. The use of the CAVE as a one-to-many presentation device at SIGGRAPH '92 and Supercomputing '92 for computational science data is also mentioned.

2004

Immersive telecommunication is a new challenging field that enables a user to share a virtual space with remote participants providing users with a sense of telepresence and interaction. This paper presents an implementation of real-time dynamic 3D avatar from multi-view cameras for the immersive telecommunication. The main objective is to offer rich communication modalities, as similar as those used in the face-to-face meetings like gestures, gaze awareness, realistic images, and correct sound direction. The proposed algorithm has been integrated to the framework of distributed VR, called NAVER, and an example of application, Heritage Alive!, has been implemented to demonstrate the concept. As a result, the user can be immersed and has natural interaction with remote participant. This would overcome the limitations both of the conventional videobased telecommunication and also the VR-based collaborative virtual environment approaches.

Proceedings of SPIE, 2000

In projection-based virtual reality systems, such as the CAVE, users can observe immersive stereoscopic images. To date, most of the images, projected onto the screens, are synthesized from polygonal models, which represent the virtual world. This is because the resolution and the viewing angle of a real image are not enough for such large screen systems. In this paper, the authors propose a novel approach to avoid the problem by exploiting the human visual systems. In the proposed system, the resolution of the center of view is very high, while that of the rest is not so high. The authors constructed a four-camera system, in which the pairs of NTSC cameras are prepared for both left and right eyes. Four video streams are combined into one video stream and captured by a graphics computer. Wide-angle multi-resolution images are synthesized in real-time from the combined video stream. Thus, we can observe the wide-angle stereoscopic video, while the resolution of the center of view is high enough. Moreover, this paper proposes another configuration of the four-camera system. Experimental results show that we can observe three levels of viewing angle and resolution by the stereoscopic effects, while images for each eye has just two levels. The discontinuities in the multi-resolution images are effectively suppressed by this new lens configuration.

Confluence of Computer Vision and Computer Graphics, 2000

In this paper we present the first implementation of a new medium for tele-collaboration. The realized testbed consists of two tele-cubicles at two Internet nodes. At each tele-cubicle a stereo-rig is used to provide an accurate dense 3D-reconstruction of a person in action. The two real dynamic worlds are exchanged over the network and visualized stereoscopically. The remote communication and the dynamic nature of tele-collaboration raise the question of optimal representation for graphics and vision. We treat the issues of limited bandwidth, latency, and processing power with a tunable 3D-representation where the user can decide over the trade-off between delay and 3D-resolution by tuning the spatial resolution, the size of the working volume, and the uncertainty of reconstruction. Due to the limited number of cameras and displays our system can not provide the user with a surround-immersive feeling. However, it is the first system that uses 3D-real-data that are reconstructed online at another site. The system has been implemented with low-cost off-the-shelf hardware and has been successfully demonstrated in local area networks.

This work presents a visual telepresence system that captures and projects panoramic images into immersive environments in real-time. A number of images are captured with a ring of cameras (a special mounting of multiple digital cameras that captures synchronized images of a 360 degrees scene). These images are processed by a PC cluster before their transmission. A panoramic image is locally rendered by the proposed optimized stitching method that computes a positioning map for all pixels. Then, a complete cylindrical image is sent over the Internet to a multi-projection virtual environment called CAVERNA Digital®. Since the panoramic view is projected in natural size onto the screens of the virtual environment, the user has the sensation of being present at the remote scene.

In this paper, a next generation 3-D video conferencing system is presented that provides immersive telepresence and natural representation of all participants in a shared virtual meeting space. The system is based on the principle of a shared virtual table environment which guarantees correct eye contact and gesture reproduction and enhances the quality of human-centered communication. The virtual environment is modeled in MPEG-4 which also allows the seamless integration of explicit 3-D head models for a low-bandwidth connection to mobile users. In this case, facial expression and motion information is transmitted instead of video streams resulting in bit-rates of a few kbit/s per participant. Beside low bit-rates, the model-based approach enables new possibilities for image enhancements like digital make-up, digital dressing, or modification of scene lighting.

1994

A new approach to telepresence is presented in which a m ultitude of stationary cameras are used to acquire both photometric and depth information. A virtual environment is constructed by displaying the acquired data from the remote site in accordance with the head position and orientation of a local participant. Shown are preliminary results of a depth image of a human subject calculated from 11 closely spaced video camera positions. A user wearing a head-mounted display walks around this 3D data that has been inserted into a 3D model of a simple room. Future systems based on this approach m a y exhibit more natural and intuitive i n teraction among participants than current 2 D teleconferencing systems.

Proceedings of the 20th annual conference on Computer graphics and interactive techniques - SIGGRAPH '93, 1993

Several common systems satisfy some but not all of the VR definition above. Flight simulators provide vehicle tracking, not head tracking, and do not generally operate in binocular stereo. Omnimax theaters give a large angle of view [8], occasionally in stereo, but are not interactive. Head-tracked monitors [4][6] provide all but a large angle of view. Head-mounted displays (HMD) [13] and BOOMs [9] use motion of the actual display screens to achieve VR by our definition. Correct projection of the imagery on large screens can also create a VR experience, this being the subject of this paper. This paper describes the CAVE (CAVE Automatic Virtual Environment) virtual reality/scientific visualization system in detail and demonstrates that projection technology applied to virtual-reality goals achieves a system that matches the quality of workstation screens in terms of resolution, color, and flicker-free stereo. In addition, this format helps reduce the effect of common tracking and system latency errors. The off-axis perspective projection techniques we use are shown to be simple and straightforward. Our techniques for doing multi-screen stereo vision are enumerated, and design barriers, past and current, are described. Advantages and disadvantages of the projection paradigm are discussed, with an analysis of the effect of tracking noise and delay on the user. Successive refinement, a necessary tool for scientific visualization, is developed in the virtual reality context. The use of the CAVE as a one-to-many presentation device at SIGGRAPH '92 and Supercomputing '92 for computational science data is also mentioned.

Proceedings of the …, 2009

We present MobileTI, a portable tele-immersive system that merges 3D video representations of users in real time to enable remote collaboration across geographical distances. With portability as a main goal, we address the challenges in the camera setup, time synchronization, video acquisition, and networking in the design and implementation of the system. Having been deployed in public performances, MobileTI proves to be effective, efficient, and user-friendly. Our experimental findings in terms of technical performance and user feedback are presented.

2001

We describe a novel approach for unifying computer graphics and computer vision systems, and our initial results in building and using a prototype system. This approach has three significant characteristics: unification of the real and virtual worlds for both input and output, tele-collaboration between remote participants, and interaction between heterogeneous stand-alone 3-dimensional (3D) graphics applications [1]. The system is designed to run on the networks of the future, and it is capable of transmitting blends of dynamic computer graphics and computer vision data in real-time. In this text we concentrate on its visual part, in particular on synergy of computer graphics and computer vision systems as a new medium for collaboration and tele-presence. The preliminary steps of our research make us increasingly optimistic that in the future this technology will provide highly compelling, immersive environments for an increasing variety of telecollaborative applications over high bandwidth networks