Modeling shapes and textures from images: new frontiers

2006

Proceedings of the IEEE Virtual Reality, 2003

A long standing goal in image-based modeling and rendering is to capture a scene from camera images and construct a sufficient model to allow photo-realistic rendering of new views. With the confluence of computer graphics and vision, the combination of research on recovering geometric structure from un-calibrated cameras with modeling and rendering has yielded numerous new methods. Yet, many challenging issues remain to be addressed before a sufficiently general and robust system could be built to eg allow an ...

ACM Transactions on Graphics, 2011

Figure 1: Photo-inspired 3D modeling of a chair from four different 3D candidates (cyan). The new models (yellow) are created as geometric variations of the candidates to fit the target object in the photo while preserving the 3D structure of the candidates.

Journal of Visualization and Computer Animation - JVCA

This paper contains two parts. In the rst part an automatic processing pipeline is pre- sented that analyses an image sequence and automatically extracts camera motion, calibra- tion and scene geometry. The system combines state-of-the-art algorithms developed in computer vision, computer graphics and photogrammetry. The approach consists of two stages. Salient features are extracted and tracked throughout the sequence to compute the camera motion and calibration and the D structure of the observed features. Then a dense estimate of the surface geometry of the observed scene is computed using stereo matching. The second part of the paper discusses how this information can be used for visualization. Traditionally, a textured D model is constructed from the computed information and used to render new images. Alternatively, it is also possible to avoid the need for an explicit D model and to obtain new views directly by combining the appropriate pixels from recorded views. It is intere...

2000

Abstract Increasingly, realistic object, scene, and event modeling is based on image data rather than manual synthesis. The paper describes a system for visits to a virtual, 3D archeological site. One can navigate through this environment, with a virtual guide as companion. One can ask questions using natural, fluent speech. The guide will respond and will bring the visitor to the desired place. Simple answers are given as changes in the orientations of his head, by him raising his eyebrows or by head nodding.

1997

In this paper, we present a new semiautomatic method to reconstruct 3D facial model for animation from two orthogonal pictures taken from front and side views. The method is based on extracting the hair and face outlines and detecting interior features in the region of mouth, eyes, etc. We show how to use structured snakes for extracting the profile boundaries and facial features. Then DFFD, a deformation process, is used for modifying a predefined or generic 3D head model to produce the individualized head. Texture mapping based on cylindrical projection is employed using a composed image from the two images. The reconstructed 3D face can be animated in our facial animation system.

The Visual Computer, 2015

This paper presents a new robust imagebased modeling system for creating high-quality 3D models of complex objects from a sequence of unconstrained photographs. The images can be acquired by a video camera or hand-held digital camera without the need of camera calibration. In contrast to previous methods we integrate correspondence-based and silhouette-based approaches, which significantly enhances the reconstruction of objects with few visual features (e.g., uni-colored objects) and improves surface smoothness. Our solution uses a mesh segmentation and charting approach in order to create a lowdistortion mesh parameterization suitable for objects of arbitrary genus. A high quality texture is produced by first parameterizing the reconstructed objects using a segmentation and charting approach, projecting suitable sections of input images onto the model, and combining them using a graph-cut technique. Holes in the texture due to surface patches without projecting input images are filled using a novel exemplar-based

2019 International Conference on 3D Vision (3DV), 2019

In this paper we report on the representation of appearance information in the context of 3D multi-view shape modeling. Most applications in image based 3D modeling resort to texture maps, a 2D mapping of shape color information into image files. Despite their unquestionable merits, in particular the ability to apply standard image tools, including compression, image textures still suffer from limitations that result from the 2D mapping of information that originally belongs to a 3D structure. This is especially true with 2D texture atlases, a generic 2D mapping for 3D mesh models that introduces discontinuities in the texture space and plagues many 3D appearance algorithms. Moreover, the per-triangle texel density of 2D image textures cannot be individually adjusted to the corresponding pixel observation density without a global change in the atlas mapping function. To address these issues, we propose a new appearance representation for image-based 3D shape modeling, which stores appearance information directly on 3D meshes, rather than a texture atlas. We show this representation to allow for input-adaptive sampling and compression support. Our experiments demonstrate that it outperforms traditional image textures, in multi-view reconstruction contexts, with better visual quality and memory footprint, which makes it a suitable tool when dealing with large amounts of data as with dynamic scene 3D models.

Procedings of the British Machine Vision Conference 2011, 2011

This paper formulates the Shape-from-Texture (SFT) problem of deriving the shape of an imaged surface from the distortion of its texture as a single-plane/multiple-view Structure-from-Motion (SFM) problem under full perspective projection. As in classical SFT formulations we approximate the surface as being piecewise planar. In contrast to many methods, our approach does not need a frontal view of the texture or the texture elements as reference, as it optimizes 3D patch positions and orientations from transformations between texture elements in the image. The reconstruction results in minimizing a large sparse linear least squares cost function based on the reprojection error, a planarity constraint and the estimated rigid motion between patches. Texture element positions in the image are estimated under the assumption of a regular texture from clustered feature points representing repeating appearances in the image. We present results obtained with synthetic data as well as real data to evaluate our method.