Project topics and materials
Get Complete Project Material File(s) Now! »
Motivation for considering human’s profile in humanrobot interaction
The importance of considering emotion as a determinant factor in human-robot interaction is the fuzzy nature of emotion classes, which may have imprecise criteria of membership. This could impose a problem when designing a human-robot interaction system based on emotion detection using the traditional recognition algorithms, which may lead the robot to generate an inappropriate behavior to the context of interaction. Therefore, in this thesis, we propose an online fuzzy-based algorithm for detecting emotion. Besides, it precises whether a new detected emotion belongs to one of the previously learnt clusters so as to get attributed to the corresponding multimodal behavior to the winner cluster, or it constitutes a new cluster that requires a new appropriate multimodal behavior to be synthesized, as discussed in Chapter (2).
On the other hand, the long term effect of personality on the verbal and nonverbal behavior of human, makes it reliable for being considered in human-robot interaction. The adaptation of the generated multimodal robot’s behavior to the extraversion-introversion personality trait of human, can increase the attraction between human and robot so as to enhance the interaction between them. Therefore, in this thesis, we examine and validate the similarity attraction principle (i.e., individuals are more attracted by others who have similar personality traits) within a human-robot interaction context. This process of interaction integrates different subsystems that allow the robot to generate an adapted synchronized multimodal behavior to human’s personality, including: a psycholinguistic-based system for detecting personality traits, and a system for generating adaptive gestures (Chapters 3 and 4).
Basic and complex emotions
Emotion is one of the most controversial issues in human-human interaction nowadays, in terms of the best way to conceptualize and interpret its role in life. It seems to be centrally involved in determining the behavioral reaction to social environmental and internal events of major significance for human [Izard, 1971; Plutchik, 1991]. One of the main difficulties behind studying the objective of emotion is that the internal experience of emotion is highly personal and is dependent on the surrounding environment circumstances. Besides, many emotions may be experienced simultaneously [Plutchik, 1991].
Different emotion theories identified relatively small sets of fundamental or basic emotions, which are meant to be fixed and universal to human (i.e., they can not be broken down into smaller parts). However, there is a deep opinion divergence regarding the number of basic emotions. Ekman [1972]; Ekman et al. [1982] stated a group of 6 fundamental emotions (i.e., anger, happiness, surprise, disgust, sadness, and fear) after studying cross-cultural facial expressions, collected from a lot of media pictures for individuals from different countries. However, Ekman in his theory had not resolved the problem discussed in the research of Izard [1971], which is the fact that it is not possible, or at least not easy, to unify basic universal facial expressions through processing media pictures only, because there are a lot of populations who have no access to media, like some populations in Africa. Consequently, there is no considerable database for their facial expressions to study. Thereafter, Izard [1977] devised a list of 10 primary emotions (i.e., anger, contempt, disgust, distress, fear, guilt, interest, joy, shame, and surprise), each one has its own neurobiological basis and pattern of expression (usually denoted by facial expressions), and each emotion is experienced uniquely. Tomkins [1984] proposed a biologically-based group of pan-cultural 9 primary emotions (i.e., anger, interest, contempt, disgust, distress, fear, joy, shame, and surprise). More theories exist in the literature of emotion modeling, similarly to the previously stated theories. However, they do not consider the evolutionary and combinatory nature of emotion, which may lead to a new advanced category of complex emotions that could be considered as mixtures of primary emotions based on cultural or idiosyncratic aspects.
Offline detection of emotional states
In this chapter, we investigated the performance of the offline classification system using the Support Vector Machine (SVM) algorithm [Cortes and Vapnik, 1995], with 15 primary and complex emotions. Afterwards, we created a fuzzy classification system and we trained it offline on 6 primary emotions, in addition to the neutral emotion (i.e., anger, disgust, happiness, sadness, surprise, fear, and neutral). However, the online test phase of the fuzzy model contained 5 complex emotions (i.e., anxiety, shame, desperation, pride, and contempt), in addition to 3 primary emotions (i.e., interest, elation, and boredom). Three databases (including more than 1000 voice sample) have been employed in training and testing the classification system. These databases are: (1) German emotional speech database (GES) [Burkhardt et al., 2005], (2) Geneva vocal emotion expression stimulus set (GVEESS) [Banse and Scherer, 1996] 1, and (3) Spanish emotional speech database (SES) [Montero et al., 1998]. 2 An important remark about the emotion classes of the total database is that they do not have all the same intensity, in addition to the existing emotion extension in two cases: boredom-disgust, and elation-happiness. This is due to the encountered difficulty to obtain well known databases with specific emotion categories that exactly match Plutchik model’s emotion categories.
Table of contents :
1 Introduction
1.1 Motivation for considering human’s profile in human-robot interaction
1.2 Robot testbeds
1.2.1 NAO Robot
1.2.2 ALICE Robot
2 An Online Fuzzy-Based Approach for Human’s Emotion Detection
2.1 Introduction
2.2 Basic and complex emotions
2.3 Offline detection of emotional states
2.3.1 Speech Signal Processing
2.3.2 Features Extraction
2.3.3 Classification
2.4 Subtractive clustering
2.5 Takagi-Sugeno (TS) fuzzy model
2.6 TS fuzzy model online updating
2.6.1 Scenario 1
2.6.2 Scenario 2
2.6.3 Scenario 3
2.7 Results and discussion
2.8 Conclusions
3 Generating an Adapted Verbal and Nonverbal Combined Robot’s Behavior to Human’s Personality
3.1 Introduction
3.2 Why should personality traits be considered in human-robot interaction?
3.3 System architecture
3.3.1 Personality Recognizer
3.3.2 PERSONAGE Generator
3.3.3 BEAT Toolkit
3.4 Extension of the nonverbal behavior knowledge base of BEAT toolkit
3.5 Modeling the synchronized verbal and nonverbal behaviors on the robot
3.6 Experimental setup
3.6.1 Hypotheses
3.6.2 Experimental Design
3.7 Experimental results
3.8 Discussion
3.9 Conclusions
4 Prosody-Based Adaptive Head and Arm Metaphoric Gestures Synthesis
4.1 Introduction
4.2 System architecture
4.3 Database
4.4 Gesture kinematic analysis
4.4.1 Linear Velocity and Acceleration of Body Segments
4.4.2 Body Segment Parameters Calculation
4.4.3 Forward Kinematics Model of the Arm
4.5 Multimodal data segmentation
4.5.1 Gesture Segmentation
4.5.2 Speech Segmentation
4.6 Multimodal data characteristics validation
4.6.1 Body Gestural Behavior Recognition in Different Emotional States
4.6.2 Emotion Recognition Based on Audio Characteristics
4.7 Data quantization
4.8 Speech to gesture coupling
4.9 Gesture synthesis validation and discussion
4.10 Conclusions
5 Multimodal Adapted Robot’s Behavior Synthesis within a Narrative Human- Robot Interaction
5.1 Introduction
5.2 System architecture
5.2.1 Metaphoric Gesture Generator
5.2.2 Affective Speech Synthesis
5.2.3 Face Expressivity
5.3 Experimental setup
5.3.1 Database
5.3.2 Hypotheses
5.3.3 Experimental Design
5.4 Experimental results
5.5 Discussion
5.6 Conclusions
6 Conclusions
