LIP FEATURES FOR PERSON RECOGNITION – Project topics materials

Get Complete Project Material File(s) Now! »

Synchronization Frame Selection

The aim of this module is to select synchronization frames from the first video of the group of videos for a specific person. Given a group of videos Vi for the person p, where i is the video index in the group, this module takes the first video V1 for each person as input and selects synchronization frames SF1 , that are considered useful for synchronization with the rest of the videos. The criterion for significance is based on amount of lip motion, hence frames that exhibit more lip motion as compared to the frames around them are considered significant. First for the video V1 the mouth region of interest (ROI) MIt for each frame t is isolated based on tracking points provided with the database. Then frame by frame optical flow is calculated using the Lucas Kanake method (cf. Figure 21) for the entire video resulting in a matrix of horizontal and vertical motion vectors. As we are interested in a general description of the amount of lip motion in the frame we then calculate the mean of the motion vectors Oft for each mouth ROI MIt.

Synchronization Frame Matching

In the previous module we have selected synchronization frames from the first video of a person and in this module we try to match these frames with the remaining videos in the group. This module can be broken down into several submodules, the first one is a feature extractor where we extracted two features related to lip motion. The second is an alignment algorithm that aligns the extracted lip features before matching, and the last sub-module is a search algorithm that matches the lip features using an adapted mean-square error algorithm. This results in the synchronization frame matrix SFi for each person.

Person recognizer module

The recognition module is similar to the one described in Chapter V.2.3, it exploits the individual feature vectors extracted from video sequences for classification purposes. The local and global features are firstly concatenated in various combinational vectors, which are subsequently used for training a Gaussian Mixture Model (GMM) for each person in the database. Classification is then performed by calculating the class conditional probability density functions in a Bayesian classifier.

Table of contents :

ABSTRACT
ACKNOWLEDGMENTS
TABLE OF INDEX
LIST OF FIGURES
LIST OF TABLE
CHAPTER I. INTRODUCTION
1. MOTIVATIONS
2. ORIGINAL CONTRIBUTIONS
3. OUTLINE
CHAPTER II. INTRODUCTION TO BIOMETRICS
1. INTRODUCTION
2. TYPES OF BIOMETRIC IDENTIFIERS
3. OPERATIONAL MODES
3.1. Verification
3.2. Identification
4. ARCHITECTURE
4.1. Enrolment
4.2. Recognition
4.3. Adaptation
5. PERFORMANCE EVALUATION
5.1. Measures for Verification
5.2. Measure for Identification
5.3. Types of Errors
6. LIMITATION AND ISSUES
6.1. Accuracy
6.2. Scale
6.3. Privacy
7. CONCLUSIONS
CHAPTER III. STATE OF ART
1. INTRODUCTION
2. PRE-PROCESSING
2.1. Speech Segmentation
2.2. Face & Lip Detection
3. FEATURE EXTRACTION
3.1. Audio Feature Extraction
3.2. Video Feature Extraction
4. CLASSIFICATION
4.1. Template Matching
4.2. Stochastic Models
4.3. Neural Networks
5. FUSION
5.1. Early Integration
5.2. Intermediate Integration
5.3. Late Integration
6. EXAMPLES OF LIP BASED PERSON RECOGNITION
6.1. Audio – Video Lip Biometric
6.2. Video only Lip Biometric
6.3. Conclusions
7. AUDIO-VIDEO SPEECH DATABASES
7.1. Introduction
7.2. VALID Database
7.3. Italian TV Database
7.4. Other Databases
CHAPTER IV. LIP DETECTION & EVALUATION
1. INTRODUCTION
2. STATE OF ART: FACE DETECTION
2.1. Feature Based Techniques
2.2. Image Based Techniques
3. STATE OF ART: LIP DETECTION
3.1. Image Based Techniques
3.2. Model Based Techniques
3.3. Hybrid Techniques
4. STATE OF ART: VISUAL LIP FEATURE
4.1. Static
4.2. Dynamic
5. PROPOSED LIP DETECTION
5.1. Edge Based Detection
5.2. Segmentation Based Detection
5.3. Error Detection and Fusion
5.4. Experiments and Results
5.5. Conclusions
6. EVALUATION OF LIP FEATURES
6.1. Introduction
6.2. Previous Work on Feature Selection
6.3. Proposed Feature Extraction
6.4. Feature Selection
6.5. Experiments and Results
6.6. Conclusions
CHAPTER V. APPLICATION OF LIP FEATURES
1. INTRODUCTION
2. LIP FEATURES FOR PERSON RECOGNITION
2.1. Introduction
2.2. Behavioural Lip Features
2.3. Person recognition
2.4. Results and experiments
2.5. Conclusions
3. LIP FEATURES FOR HCI
3.1. Introduction
3.2. Head gesture recognition
3.3. Lip Reading
3.4. Conclusions
4. LIP FEATURES FOR GENDER
4.1. Related Works
4.2. Proposed Method
4.3. Experiments and Results
4.4. Conclusion
CHAPTER VI. LIP FEATURE NORMALIZATION
1. INTRODUCTION
2. SYNCHRONIZATION
2.1. Synchronization Frame Selection
2.2. Synchronization Frame Matching
2.3. Person Recognition
2.4. Experiments and Results
2.5. Conclusions
3. NORMALIZATION
3.1. Optimal Number of Frames.
3.2. Transcoding
3.3. Person recognition
3.4. Experiments and results
3.5. Conclusions
CHAPTER VII. CONCLUSIONS
1. CONCLUDING SUMMARY
2. FUTURE WORKS
3. PUBLICATIONS
CHAPTER VIII. APPENDICES
FACE AND EYE FEATURES FOR PERSON RECOGNITION
1. FACIAL FEATURE EXTRACTION
1.1. Face Angle
1.2. Face Symmetry
2. EYE DYNAMICS
3. PERSON RECOGNIZER MODULE
4. EXPERIMENTAL RESULTS AND DISCUSSIONS
5. CONCLUSIONS AND FUTURE WORKS
REFERENCES