Project topics and materials
Get Complete Project Material File(s) Now! »
Overview of Multi-view stereo
Multi-view stereo can be considered as the inverse process of taking photographs of a fixed scene. While a photograph is the map of the 3D scene into a 2D domain through a camera, multi-view stereo has the goal of recovery of this scene from its multiple photographs in different points of view. While there are many camera models, the standard model is a the pinhole camera (Fig. 2.1). In this model, the light from the world traverse the optical center and meets the image sensor following a straight line. Men and animals estimate the 3-dimension shapes and depth in relation with the position and the direction of their eyes. Likewise, the 3D reconstruction from photographs (images) relies on the knowledge of position and orientation of the cameras. Therefore, the multi-view stereo usually goes after the estimation of cameras parameters. Such estimation is called ‘camera calibration’, which is not in the scope of this thesis. We refer interested readers to an excellent textbook [Hartley and Zisserman, 2004].
Matching pixels through multiple images taken from different known views provide 3D positions of the object. Effectively, a 3D point A on the surface of the object is projected to a position a1 in the captor of a camera 1, and position a2 in the captor of a camera 2 (Fig. 2.2). Then knowing the camera parameters, such as their optical centers c1 and c2, we can compute the position of point A as the intersection of two rays c1a1 and c2a2. It is called the ’triangulation’.
Framework in photo-consistency and regularization measures
Multi-view stereo is based on the similarity and the matching among pixels (or sub-pixels) of different images. The similarity of pixels depends on some photoconsistency of their values. Moreover, we expect the depth map of an image is quite continuous (except in occlusion area), and the object surface is rather smooth. The reconstruction then needs another measure, which is called the regularization. Formulating in Bayesian approach, given I the set of images, we need to find the surface S such that the probability P(S|I) is maximized. We have: P(S|I) = P(S)P(I|S)/P(I). The term P(I|S) can be considered as data (photoconsistency) term, and P(S) as regularization term (the probability of the surface occurs independent to images). While not all methods formulate in a Bayesian approach, most of them use the regularization for superior quality results.
Transformation steps in multi-view stereo methods
We consider a full multi-view stereo method is a process that transform images information (with their camera matrices) into a watertight surface. During this transformation, many intermediate representations and algorithms could be used. We will describe different methods in intermediate steps: ‘images→discrete presentation’, ‘discrete presentation → surface’, ‘surface → surface’, ‘volume → surface’. A discrete presentation consists of point set, set of patches, depth maps (or range images). A volume is a volume containing the scene, like a grid box, or a visual hull. Some transformation like ‘discrete presentation → surface’, or ‘surface → surface’, could use a volume in their algorithms, but we do not include them in the ‘volume → surface’ part. A transformation could be used many time, not necessary consecutively, to improve the quality of the results.
Table of contents :
1 Introduction (version fran¸caise)
1.1 Conception g´en´erale de multi-vues st´er´eo
1.2 Le sujet de th`ese et les contributions
1.2.1 Liste de publications
2 Introduction
2.1 Overview of Multi-view stereo
2.2 Thesis subject and contribution
2.2.1 List of publiations
3 Review on Multi-view stereo
3.1 Common concepts
3.2 Framework in photo-consistency and regularization measures
3.2.1 Photo-consistency matching
3.2.2 Regularization
3.3 Transformation steps in multi-view stereo methods
3.3.1 From images to a discrete presentation
3.3.2 From a discrete presentation to a surface
3.3.3 From a surface to a surface
3.3.4 From a volume to a surface
3.4 Large scale multi-view stereo
3.4.1 Multi-view stereo for compact objects
3.4.2 Multi-view stereo for outdoor scenes
3.4.3 3D reconstruction on Internet scale
3.5 Some related topics
3.5.1 Structure from motion
3.5.2 Active range finding
3.5.3 Shape from X
3.5.4 Reconstruction of dynamic scenes
3.5.5 Urban architecture understanding
3.6 Conclusion
4 Towards large-scale multi-view stereo
4.1 Introduction
4.1.1 Motivation
4.1.2 Contributions
4.2 Multi-view reconstruction pipeline
4.2.1 Quasi-dense point cloud
4.2.2 Visibility-based surface reconstruction
4.2.3 Photometric robust variational refinement
4.2.4 Discretization
4.3 Implementation aspects
4.4 Experimental results
4.4.1 Compact objects
4.4.2 Outdoor architectural scenes
4.4.3 Landscape and cultural heritage scenes
4.5 Conclusion
5 Surface triangular mesh merging
5.1 Introduction
5.1.1 Contribution
5.2 Merging algorithm in general case
5.2.1 Overlap detection
5.2.2 Graph cuts on Constrained Delaunay Tetrahedralization
5.2.3 Graph cuts on the extracted surface
5.2.4 Experiments
5.3 Merging meshes from partition of bounding box
5.4 Conclusion
6 Large-scale visibility-consistent surface reconstruction
6.1 Introduction
6.1.1 Motivation
6.1.2 Work in multi-view stereo with the visibility issue
6.1.3 Work on large scale surface reconstruction
6.2 Divide and Conquer algorithm
6.2.1 Multi-level representation of a point set
6.2.2 Partition of a point set in many equal parts
6.2.3 Local visibility-consistent surface reconstruction
6.2.4 Multi-level point cloud filter
6.2.5 Partial surface reconstruction and mesh merging
6.3 Experiments
6.4 Limitation
6.5 Conclusion
7 Large scale multi-view stereo
7.1 Introduction
7.2 Partition of mesh with associated images
7.3 Sequential and independent deformations
7.4 Experiments
7.5 Some comparison with PMVS
7.6 Conclusion
8 Conclusion
A Background
A.1 Delaunay triangulation
A.2 Constrained Delaunay triangulation
A.3 Graph cuts optimization
A.4 kd-tree
A.5 kd-tree of volumetric objects
B Supplement results
Bibliography
