Volumetric stereo with silhouette and feature constraints

2007 IEEE 11th International Conference on Computer Vision, 2007

We formulate multi-view 3D shape reconstruction as the computation of a minimum cut on the dual graph of a semiregular, multi-resolution, tetrahedral mesh. Our method does not assume that the surface lies within a finite band around the visual hull or any other base surface. Instead, it uses photo-consistency to guide the adaptive subdivision of a coarse mesh of the bounding volume. This generates a multi-resolution volumetric mesh that is densely tesselated in the parts likely to contain the unknown surface. The graph-cut on the dual graph of this tetrahedral mesh produces a minimum cut corresponding to a triangulated surface that minimizes a global surface cost functional. Our method makes no assumptions about topology and can recover deep concavities when enough cameras observe them. Our formulation also allows silhouette constraints to be enforced during the graph-cut step to counter its inherent bias for producing minimal surfaces. Local shape refinement via surface deformation is used to recover details in the reconstructed surface. Reconstructions of the Multi-View Stereo Evaluation benchmark datasets and other real datasets show the effectiveness of our method.

Image and Vision Computing, 2010

2010

This paper presents a volumetric stereo and silhouette fusion algorithm for acquiring high quality models from multiple calibrated photographs. Our method is based on computing and merging depth maps. Different to previous methods of this category, the silhouette information is also applied in our algorithm to recover the shape information on the textureless and occluded areas. The proposed algorithm starts by computing visual hull using a volumetric method in which a novel projection test method is proposed for visual hull octree construction. Then, the depth map of each image is estimated by an expansion-based approach that returns a 3D point cloud with outliers and redundant information. After generating an oriented point cloud from stereo by rejecting outlier, reducing scale, and estimating surface normal for the depth maps, another oriented point cloud from silhouette is added by carving the visual hull octree structure using the point cloud from stereo to restore the textureless and occluded surfaces. Finally, Poisson Surface Reconstruction approach is applied to convert the oriented point cloud both from stereo and silhouette into a complete and accurate triangulated mesh model. The proposed approach has been implemented and the performance of the approach is demonstrated on several real datasets, along with qualitative comparisons with the state-of-the-art

2012

We propose a novel method on stereo matching based on the Global Edge Constraint (GEC) and Graph Cuts. Firstly, the GEC composed of particular image edges is employed to generate the initial disparity maps. And then the reliable disparity maps consistent with the observed data are extracted to construct the data term of the energy function. Finally, we incorporate the GEC as a soft constraint into our global optimization framework, and the optimal solution could be approximated via graph cuts. Experimental results demonstrated the good performance of our proposed approach.

2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05), 2005

Occlusion is usually modelled in two images symmetrically in previous stereo algorithms which cannot work for multi-view stereo efficiently. In this paper, we present a novel formulation that handles occlusion using only one depth map in an asymmetrical way. Consequently, multiview information is efficiently accumulated to achieve high accuracy. The resulting energy function is complex and approximate graph cut based solutions are proposed. Our approach complements the theory and extends the applicability of using graph cut in stereo. The experiments demonstrate that the approach is comparable with the state of the art and potentially more efficient for multi-view stereo.

2008 IEEE Conference on Computer Vision and Pattern Recognition, 2008

Recovering the three-dimensional (3D) object shape remains an unresolved area of research on the cross-section of computer vision, photogrammetry and bioinformatics. Although various techniques have been developed, the computational complexity and the constraints introduced to overcome the problems have limited their applicability in the real world scenarios. In this paper, we propose a method that is based on the projective geometry between the object space and the silhouette-images taken from multiple viewpoints. The approach eliminates the problems related to dense feature point matching and camera calibration that are generally adopted by many state of the art shape reconstruction methods. The object shape is reconstructed by establishing a set of hypothetical planes slicing the object volume and estimating the projective geometric relations between the images of these planes. The experimental results show that the 3D object shape can be recovered by applying minimal constraints.

Studies in Computational Intelligence, 2010

Acquiring 3d shape from images is a classic problem in Computer Vision occupying researchers for at least 20 years. Only recently however have these ideas matured enough to provide highly accurate results. We present a complete algorithm to reconstruct 3d objects from images using the stereo correspondence cue. The technique can be described as a pipeline of four basic building blocks: camera calibration, image segmentation, photo-consistency estimation from images, and surface extraction from photo-consistency. In this chapter we will put more emphasis on the latter two: namely how to extract geometric information from a set of photographs without explicit camera visibility, and how to combine different geometry estimates in an optimal way. recovering the geometry just from photographs is not only a challenging problem but also, in the general case, an ill-posed problem. It is challenging because lighting and material properties play a very important role in the image formation model. The same geometry with a different material or different lighting conditions can give extremely different photographs. It is also an ill-posed problem because, in the general case, different combinations of geometry, lighting and material can produce exactly the same photographs, making it impossible to recover a single scene geometry. The main recipe to make the problem well-posed is to use priors on the types of surface that one expects. Traditionally the most common type of prior is the smooth surface prior. However when dealing with special classes of objects such as human faces or man-made objects, more evolved priors have been successfully used, e.g. human faces [54], buildings or planes .

2006

This paper presents an optimization framework for the 3D reconstruction of the surfaces from stereo image pairs. The method is based on employing popular graph cut methods under the dual mesh optimization technique. The constructed system produces noticeably better results by running two separate optimization processes that communicate with each other. The communication mechanism makes our system more robust against local minima and it produces extra side information about the scene such as the unreliable image sections. We validated our system by running experiments on real data with ground truth and we compared our results with the other optimization methods, which showed the accuracy and effectiveness of our method.