Interactive Zoom and Panning from Live Panoramic Video

2002

This paper describes a novel solution to problems associated with interactive display of immersive stereographic imagery via Apple's QuickTime Virtual Reality (QTVR) technology. A unique multinode implementation providing Stereographic QTVR (termed here SQTVR) enables the display of pairs of images exhibiting binocular parallax, and the stereoscopic depth percept that results is enhanced by motion parallax inherent in a subtle translation of the viewpoint through the displayed 3D scene. Stereoscopic depth is maintained as a user pans freely through a complete ¢ ¤ £ ¦ ¥ § horizontal panorama, while the system imposes a slight dollying in and out of the scene as a user's view rotates left or right. In addition, SQTVR solves two problems that can be observed when users of conventional QTVR technology change viewing positions, and these are problems that generally interfere with a user's sense of immersion and telepresence. First, objects that should be revealed ("disoccluded") by a change in viewing position remain occluded behind objects in the foreground of the image. Second, objects that should loom large as they approach the viewing position are displayed in an unchanging proportion relative to image elements in the background. SQTVR's multinode implementation addresses these two limitations of conventional QTVR technology in a natural way by tiling a plane with equilateral triangles connecting potential viewing positions.

Proceedings of the International Conference on Signal Processing and Multimedia Applications, 2009

Panoramic immersive video is a new technology, which allows the user to interact with a video beyond simple production line, because it enables the possibility to navigate in the scene from different points of view. Although many devices for the production of panoramic videos have been proposed, these are still expensive.In this paper a framework for production of virtual panoramic immersive videos using 3D production software is presented. The framework is composed by two stages: panoramic video production and immersive visualization. In the former stage the traditional 3D scene is taken as input and two outputs are generated, the panoramic video and path sounds to immersive audio reproduction. In the latter, a desktop CAVE assembly is proposed in order to provide an immersive display.

IEEE Transactions on Consumer Electronics, 2011

This paper deals with a real-time panorama algorithm for mobile camera systems. The proposed system generates the panoramic images and shows the intermediate results simultaneously on the mobile display. The proposed panorama system consists of feature point extraction, feature tracking, rotation matrix estimation, and image warping onto cylindrical surface. Feature points are extracted by fast Hessian detector in the low resolution and corner detector in the high resolution. Then, the feature points are tracked by matching between input frame and synthesized panorama image. The camera motion is modeled as a rotation matrix which is estimated using the tracked feature points. For realtime operation of panoramic image synthesis, we propose a method to estimate the rotation matrix using non-iterative least square method. We project the feature points of image onto the unit sphere surface in the world coordinates. This enables us to estimate the parameters of rotation matrix as the linear and non-iterative problem. Finally, we project the input frames onto panorama surface using the rotation matrix. We also implement a real-time display interface to show the intermediate panorama results while panorama image is being generated. Thus, the proposed panorama system paints the panorama canvas with real images. According to the experiments in the mobile systems such as mobile phone and tablet PC, the proposed system works well to generate the panoramic images in real-time.

2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops, 2010

We introduce a mobile system for creating highresolution panoramic images. The user can rotate the camera around an arbitrary axis to create a 2D sweep and see a miniature preview panorama in real-time. The system tracks camera motion and automatically captures high-resolution images and generates a high-quality wide-view panoramic image. We employ a coarse-to-fine method for high-quality registration, and a seam-finding method to remove ghosting effects due to moving objects. The proposed system has been tested on several camera phones, and the tests reveal that the system can efficiently provide a high-quality panoramic image in spite of the low computational power and memory available in such devices.

Lecture Notes in Computer Science, 2003

Techniques for synthesizing panoramic scenes are widespread. Such a scene can be automatically created from multiple displaced images by aligning and overlapping them using an image registration technique. The ability to generate panoramic scenes has many applications including the generation of virtual reality backgrounds, model-based video compression, and object recognition. These techniques-and consequently their associated applications share the restriction that all scenes are limited to a 360 degree view of the horizontal plane at the particular moment in time the images were taken. Until recently, there has been little motivation to develop techniques for the presentation of complete spherical views in real time-scenes that present the entire potential visible fields of view, through time. With the advent of space exploration and associated micro-gravity environments, "up" and "down" are relative terms and locally fixed points of reference are difficult to come by. It may be useful to rethink how video is captured and presented to a user working in such an environment employing extended notions of what a panorama is. The Panoramic Spherical Video (PSV) system described in this paper allows a user to view and pan through arbitrary angles of view including elevation and declination as well as providing the view in real time from an array of 16 synchronized CCD video cameras whose video output are selectively "stitched" together to provide a smooth transition between camera fields of view. In this way, the user can smoothly pan through all the fields of view that are generated by the system. All video processing is done in software-there are no moving parts.

We present a practical solution for generating 360 degrees stereo panoramic videos using a single camera. Current approaches either use a moving camera that captures multiple images of a scene, which are then stitched together to form the final panorama, or use multiple cameras that are synchronized. A moving camera limits the solution to static scenes, while multi-camera solutions require dedicated calibrated setups. Our approach improves upon the existing solutions in two significant ways: It solves the problem using a single camera, thus minimizing the calibration problem and providing us the ability to convert any digital camera into a panoramic stereo capture device. It captures all the light rays required for stereo panoramas in a single frame using a compact custom designed mirror, thus making the design practical to manufacture and easier to use. We analyze several properties of the design as well as present panoramic stereo and depth estimation results.

… (VR), 2010 IEEE, 2010

We present a novel method for the real-time creation and tracking of panoramic maps on mobile phones. The maps generated with this technique are visually appealing, very accurate and allow drift-free rotation tracking. This method runs on mobile phones at 30 Hz and has applications in the creation of panoramic images for offline browsing, for visual enhancements through environment mapping and for outdoor Augmented Reality on mobile phones.

ijcsit.com

– Panoramic image stitching is the process of combining multiple photographic images with overlapping fields of view to produce a panorama. First, salient features are robustly detected from the input images by a robust algorithm called Scale Invariant Feature Transform ...

1995

Traditionally, virtual reality systems use 3D computer graphics to model and render virtual environments in real-time. This approach usually requires laborious modeling and expensive special purpose rendering hardware. The rendering quality and scene complexity are often limited because of the real-time constraint. This paper presents a new approach which uses 360-degree cylindrical panoramic images to compose a virtual environment. The panoramic image is digitally warped on-the-fly to simulate camera panning and zooming. The panoramic images can be created with computer rendering, specialized panoramic cameras or by "stitching" together overlapping photographs taken with a regular camera. Walking in a space is currently accomplished by "hopping" to different panoramic points. The image-based approach has been used in the commercial product QuickTime VR, a virtual reality extension to Apple Computer's QuickTime digital multimedia framework. The paper describes the architecture, the file format, the authoring process and the interactive players of the VR system. In addition to panoramic viewing, the system includes viewing of an object from different directions and hit-testing through orientation-independent hot spots.

2011

Constructing a panoramic video out of multiple incoming live mobile video streams is a challenging problem. This problem involves multiple users live streaming the same scene from different angles, using their mobile phones, with the objective of constructing a panoramic video of the scene. The main challenge in this problem is the lack of coordination between the streaming users, resulting in too much, too little, or no overlap between incoming streams. To add to the challenge, the streaming users are generally free to move, which means that the amounts of overlap between the different streams are dynamically changing. In this paper, we propose a method for automatically coordinating between the streaming users, such that the quality of the resulting panoramic video is enhanced. The method works by analyzing the incoming video streams, and automatically providing active feedback to the streaming users. We investigate different methods for generating the active feedback and presenting it to the streaming users resulting in an improved panoramic video output compared to the case where no feedback is utilized.