Project topics and materials
Get Complete Project Material File(s) Now! »
Generic reconstruction methods
The most common manner to retrieve the 3D geometry of objects and scenes using image is through image-based two-view reconstruction. The idea is rather simple : under the assumption that we can observe the projection of a 3D point in two different images taken from different view points, it’s possible to determine a measurement that associates the observed images with the relative depth of this point. The same idea is the basis of multi-view reconstruction methods where additional views contribute to the reconstruction process through additional constraints. In order to review the two-view stereo-reconstruction methods, one first has to introduce some basic notions of image-based 3D geometry.
3D Face Reconstruction for Facial Animation
As mentioned in the state of the art, 3D face reconstruction can be achieved in many different ways. Our goal is to reconstruct face with the purpose to animate it. To this end, one should be able to have a parametric face model, (with animation parameters) representing the 3D geometry of the face. Model fitting [43, 49] appears to be the best way to reach this objective. To this end, we need to register a face model, to the 3D surface of the face. First, we will present our work on 3D face surface reconstruction, and then our animation face model and the registration technique to fit it on the surface.
Several approaches, which proved themselves, are available for surface reconstruction. For model fitting technique, there is no need of advance reconstruction, since it’s the face model itself, which gives the human aspect of the reconstruction. Combinatorial methods are particularly suitable for this task, as the reconstruction process, in the case of rectified stereo images is very simple, and because the problem is directly defined as a discreet problem (compared to variational and level sets methods for which there is a need to reformulate the problem in the continuous domain). But this aspect is also a draw back when the reconstruction is based on low resolution images. We propose a Super Resolution Reconstruction [44], to improve the reconstruction.
Stereo Reconstruction and Graph Cut
Let us now introduce in details some notions from combinatorial optimization, namely the graph cut approach. Let G be a graph, consisting of a set of nodes V and a set of directed edges E that connect them such as G = (V; E) (See [Fig. (1.8).i]). The nodes set V contains two special terminal nodes which are called the source, s, and the sink, t, while the edges set E is divided in two sub-set : t-links for terminal links, linking terminal nodes with other nodes, and n-links for neighborhood links, linking non-terminal nodes together. All edges in the graph are assigned some non-negative weight or cost. A cut C is a partitioning of the nodes in the graph into two disjoint subsets S and T such that the source s is in S and the sink t is in T. The cost of a cut C = (S; T) (See [Fig. (1.8).ii]) is defined as the sum of the costs of boundary edges (p; q) where p 2 S and q 2 T. The minimum cut problem on a graph is to find a cut that has the lowest cost among all possible cuts. One of the fundamental results in combinatorial optimization is that the minimum cut problem can be solved by finding a maximum flow from the source s to the sink t.
Table of contents :
Introduction
Introduction (en français)
1 Face Reconstruction and Face Modeling
1.1 Introduction
1.2 Facial Reconstruction and State of the Art
1.2.1 Material Based Reconstruction
Laser range scanning
Structured light projection
1.2.2 Generic reconstruction methods
Reminder on 3D Geometry
Variational and level sets methods
Combinatorial methods
Space Carving
Structure from Motion
1.2.3 Specific Reconstruction methods
Model-based Facial Reconstruction Methods
Bundle Adjustment
Model-based Facial Reconstruction and Structure from
Motion
Active Appearance Models
Morphable Facial Models
Facial Reconstruction, Priors and Image/Model-based approaches
Mixed reconstruction methods
1.3 3D Face Reconstruction for Facial Animation
1.3.1 Stereo Reconstruction and Graph Cut
Redefinition of Local Consistency toward Exploiting Facial
Geometry
Super Resolution Image Reconstruction
Super Resolution Method
1.4 Face Model
1.4.1 Candide Face Model
1.4.2 Candide Improvements
1.4.3 Face Model Animation
State of the art
1.4.4 Animation process of our model
1.5 Conlusion
2 Face Inference
2.1 Preliminary work : Face Detection
Integral Image
Haar Basis Functions
Adaboost Algorithm
Fast selection of critical features
Combination of classifiers in cascade
2.2 Facial Features Extraction
2.2.1 State of the Art
Texture-based Method
Shape-based Method
Hybrid Method
2.3 Anthropometric constraints for facial features extraction
2.3.1 Markov Random Filed formulation of the problem
2.3.2 Optimization process through Fast-PD
2.4 3D Pose Estimation from a single image
2.4.1 The Prior Constraints
2.4.2 The Pose Estimation
Shape-driven Pose Estimation
Image-driven Pose Estimation
Optimization
2.5 Motion Tracking
2.5.1 Features tracking
2.5.2 Model Based tracking
2.5.3 3D Feature Points Tracking
2.6 Conclusion
3 Facial Behavior Analysis
3.1 Emotion Modeling
3.1.1 State of the Art
3.1.2 Expression Modeling as a Time Serie
3.1.3 3D Database of Facial expression
3.1.4 Expressions Modeling on our Face Model
3.2 Emotion Recognition
3.2.1 Hidden Markov Model
3.2.2 Support Vector Machine
3.2.3 Neural Network
3.2.4 Adaboost
3.2.5 Belief Propagation
3.2.6 Rule Based
3.2.7 Distance to a model
3.3 Conclusion
Conclusion
