Get Complete Project Material File(s) Now! »

## Generic reconstruction methods

The most common manner to retrieve the 3D geometry of objects and scenes using image is through image-based two-view reconstruction. The idea is rather simple : under the assumption that we can observe the projection of a 3D point in two different images taken from different view points, it’s possible to determine a measurement that associates the observed images with the relative depth of this point. The same idea is the basis of multi-view reconstruction methods where additional views contribute to the reconstruction process through additional constraints. In order to review the two-view stereo-reconstruction methods, one first has to introduce some basic notions of image-based 3D geometry.

### 3D Face Reconstruction for Facial Animation

As mentioned in the state of the art, 3D face reconstruction can be achieved in many different ways. Our goal is to reconstruct face with the purpose to animate it. To this end, one should be able to have a parametric face model, (with animation parameters) representing the 3D geometry of the face. Model fitting [43, 49] appears to be the best way to reach this objective. To this end, we need to register a face model, to the 3D surface of the face. First, we will present our work on 3D face surface reconstruction, and then our animation face model and the registration technique to fit it on the surface.

Several approaches, which proved themselves, are available for surface reconstruction. For model fitting technique, there is no need of advance reconstruction, since it’s the face model itself, which gives the human aspect of the reconstruction. Combinatorial methods are particularly suitable for this task, as the reconstruction process, in the case of rectified stereo images is very simple, and because the problem is directly defined as a discreet problem (compared to variational and level sets methods for which there is a need to reformulate the problem in the continuous domain). But this aspect is also a draw back when the reconstruction is based on low resolution images. We propose a Super Resolution Reconstruction [44], to improve the reconstruction.

#### Stereo Reconstruction and Graph Cut

Let us now introduce in details some notions from combinatorial optimization, namely the graph cut approach. Let G be a graph, consisting of a set of nodes V and a set of directed edges E that connect them such as G = (V; E) (See [Fig. (1.8).i]). The nodes set V contains two special terminal nodes which are called the source, s, and the sink, t, while the edges set E is divided in two sub-set : t-links for terminal links, linking terminal nodes with other nodes, and n-links for neighborhood links, linking non-terminal nodes together. All edges in the graph are assigned some non-negative weight or cost. A cut C is a partitioning of the nodes in the graph into two disjoint subsets S and T such that the source s is in S and the sink t is in T. The cost of a cut C = (S; T) (See [Fig. (1.8).ii]) is defined as the sum of the costs of boundary edges (p; q) where p 2 S and q 2 T. The minimum cut problem on a graph is to find a cut that has the lowest cost among all possible cuts. One of the fundamental results in combinatorial optimization is that the minimum cut problem can be solved by finding a maximum flow from the source s to the sink t.

**Table of contents :**

Introduction

Introduction (en français)

**1 Face Reconstruction and Face Modeling **

1.1 Introduction

1.2 Facial Reconstruction and State of the Art

1.2.1 Material Based Reconstruction

Laser range scanning

Structured light projection

1.2.2 Generic reconstruction methods

Reminder on 3D Geometry

Variational and level sets methods

Combinatorial methods

Space Carving

Structure from Motion

1.2.3 Specific Reconstruction methods

Model-based Facial Reconstruction Methods

Bundle Adjustment

Model-based Facial Reconstruction and Structure from

Motion

Active Appearance Models

Morphable Facial Models

Facial Reconstruction, Priors and Image/Model-based approaches

Mixed reconstruction methods

1.3 3D Face Reconstruction for Facial Animation

1.3.1 Stereo Reconstruction and Graph Cut

Redefinition of Local Consistency toward Exploiting Facial

Geometry

Super Resolution Image Reconstruction

Super Resolution Method

1.4 Face Model

1.4.1 Candide Face Model

1.4.2 Candide Improvements

1.4.3 Face Model Animation

State of the art

1.4.4 Animation process of our model

1.5 Conlusion

**2 Face Inference **

2.1 Preliminary work : Face Detection

Integral Image

Haar Basis Functions

Adaboost Algorithm

Fast selection of critical features

Combination of classifiers in cascade

2.2 Facial Features Extraction

2.2.1 State of the Art

Texture-based Method

Shape-based Method

Hybrid Method

2.3 Anthropometric constraints for facial features extraction

2.3.1 Markov Random Filed formulation of the problem

2.3.2 Optimization process through Fast-PD

2.4 3D Pose Estimation from a single image

2.4.1 The Prior Constraints

2.4.2 The Pose Estimation

Shape-driven Pose Estimation

Image-driven Pose Estimation

Optimization

2.5 Motion Tracking

2.5.1 Features tracking

2.5.2 Model Based tracking

2.5.3 3D Feature Points Tracking

2.6 Conclusion

**3 Facial Behavior Analysis **

3.1 Emotion Modeling

3.1.1 State of the Art

3.1.2 Expression Modeling as a Time Serie

3.1.3 3D Database of Facial expression

3.1.4 Expressions Modeling on our Face Model

3.2 Emotion Recognition

3.2.1 Hidden Markov Model

3.2.2 Support Vector Machine

3.2.3 Neural Network

3.2.4 Adaboost

3.2.5 Belief Propagation

3.2.6 Rule Based

3.2.7 Distance to a model

3.3 Conclusion

**Conclusion**