Immersion and Presence Factors in Virtual Environments
Immersion and presence are two terms currently used when talking about virtual reality. Even if related, these two terms should not be confused. Immersion is about the technology. Vianin de-fined it as the technical interface between the man and the machine not concerning psychological states of the user [Vianin1995]. Eﬀectively, immersion concerns the interaction of the subject with the virtual environment. These interactions are possible throughout sensorial information (vision, audition, smell, kinesthetic, . . . ). These interactions, necessary for the human-machine coupling are created throughout behavioral interfaces. These interfaces are divided in two cate-gories, motor interfaces (motion capture, sensors, computer vision, . . . ) and sensorial interfaces (referring to the five senses, like screen and 3D glasses, for example). Fuchs et Al. [Fuchs2006] proposed a third category concerning sensorimotor interfaces like, for example, force feedback interfaces. The quality of these interfaces will influence the sensation of immersion in the virtual environment of the subject. What diﬀerentiates immersion from presence, that will be defined next, is that in immersion the psychological aﬀect of the subject is not involve.
On the other hand, presence can be defined as the psychological sensation of being in an environment created using immersion technologies. That sensation of presence is not equally induced for all user of a same immersive environment [Slater1993]. That definition was subject to many variations. Barfield and Weghorst [Barfield1993] talk about virtual presence and defined it as a subjective and hypothetical conscious state of being implied in a non-present environment. Steuer [Steuer1992] instead talk about « telepresence » and defined it as the experience of a presence in an environment mediated by a communication mean. In that case, presence refer to natural perception of an environment as telepresence refer the perception of a mediated environment.
The sensation of presence is not unique to virtual environments. According to Hendrix [Hendrix1994], presence can also appear when reading a book, watching a movie or a theater act. Presence is therefore not something new. However, a high degree of presence is something important to reach in virtual reality applications.
Slater and Usoh [Slater1993] classified the presence factors in two categories. The first category concerns the external factor and is closely link with the immersion technology used to create the virtual environment. The second category concerns the user internal state.
The external factors are related to the material used to create the virtual environment. According to Steuer [Steuer1992] there is five important points to consider:
• the quality and richness of the sensorial information transmitted by the environment;
• the coherence of the virtual environment;
• the degree of interaction between the subject and the environment;
• the fidelity of the reproduction of the subject virtual body (avatar);
• the latency of the system (as close to zero as possible) between the subject action and the reaction cause in the virtual environment.
The internal factors are related to the internalization processes of the users. These factors con-sequently largely varied from one subject to another [Psotka1995]. That finding was confirmed by a study by Hodgins et Al. [Hodgins1998] analyzing the influence of geometric models on perceptions. That study showed that for some users the action felt more realistic on a textured polygonal model than on a stick figure model as other users did not perceived any diﬀerence.
The level of presence is therefore highly dependent on the individual, especially if the graphical quality of the environment is low. The presence level depends on the subject implication and on his capacity to supplement the virtual environment with his own representation of the world. However, it seems possible to study presence on a population and to draw general conclusion about the capacity of a virtual environment to induce presence.
Presence Measure Methods
Presence is a multidimensional complex phenomena and it is consequently not possible to mea-sure it using only one kind of measures. Furthermore, subjective measures are not always reliable and pose a validity problem [Barfield1993]. The notion of presence itself is not perfectly defined. However, many methodologies to measure it have been created, and Hendrix divided these into objective and subjective measures [Hendrix1994].
Regarding the objective measures, Barfield and Weghorts [Barfield1993] outlined some cate-gories:
• Physiologic indicators: muscular tension, cardiovascular and ocular response, skin conduc-tion, . .
• Physiometric indicators: cortical responses, pupil diameter, . . .
• Performance indicators: precision, speed, . . . [Slater1996]
• Conflict resolution indicators: the capacity of the subject to adapt to the conflictual situations generated by the presence of two environments, the physical world and the virtual environment.
• Subject disorientation degree indicators: time to re-adapt to the physical world.
Subjective measures mostly rely on questionnaires to evaluate the degree of presence of an individual in a virtual environment [Witmer1998, Usoh2000, Slater1999]. Questionnaires alone cannot measure all factors influencing presence, it is therefore important to combine subjective and objective measures [Hendrix1994]. Furthermore, Slater et Al. have point out the diﬃculty of formalizing questions to assess presence of the virtual experience of a participants [Slater2007] [Slater2004].
Presence and Performance
The study of the relationship between presence and performance has major impact in virtual reality. Knowing the presence factors that can possibly improved the performance of a task execution in teleoperation has a direct impact on the conception of these environments. To establish the link between performance and presence, it is necessary to identify a list of behavioral interfaces that improved the performance of a specific task [Draper1996]. The problem is not to know if presence influences the performance, but to determine the combination of media required to induce a reaction similar to that that would be induced in real life situation [Slater1996]. If that combination is known it is be possible to train individual in virtual reality to prepare them to be performant in real life situations. For example, to ameliorate the eﬃciency of a surgeon in the operation room [Freysinger2002]. However, that is not an absolute rule. In some cases, a high level of presence might induce a dependance to the virtual environment to perform the task. Situations were transfer from one environment to another are required should consequently be carefully study. Sections
resents more details about that.
However, establishing the level of performance in virtual environments, gives information about the level of presence. Consequently, if an individual correctly performed a task in virtual reality, that implies that his presence and implication level were suﬃcient [Slater1999]. In fact, the definition of presence (being there) implies « being able to act there » [Sanchez-Vives2005].
Furthermore, performance and presence can also be related in another manner. The eﬃciency of a simulation can be evaluated by its degree of fidelity [Stroﬀregen2002]. That degree of fidelity being defined by the diﬀerence between the subject behavior in the virtual environment and the real world situation. System fidelity can therefore be divided in two categories: the fidelity of the experience, and the fidelity of the action performed in the virtual environment. The first category refers to the notion of presence. The second category is a measure of the functional fidelity. Functional fidelity is considered high if action performed in the virtual environment is close to that that would be performed in the real world [Morice2008].
The notion of performance is extremely important in the domain of sports. Bideau et Al. [Bideau2003, Bideau2004a] and later Vignais et Al. [Vignais2009a, Vignais2010] have studied the reaction of handball goalkeeper to virtual throwers in order to establish if the level of performance was the same in virtual reality than in a real situation. They were able to establish that the level of performance, and consequently of presence, was suﬃcient to use virtual reality as a study tool for the visual information retrieval activity of an handball goalkeeper.
Body ownership can be seen as special cases of presence, where the presence to evaluate is that of the subject in a virtual human present in the virtual environment. That virtual human is a projection of the subject in the virtual environment and can be seen either in first person view, normally throughout the usage of and HMD where one can look at himself (a virtual self), or in third person view, often by being represented in the virtual environment throughout the usage of a mirror or shadow metaphor.
Body ownership is a known phenomena outside the virtual reality context and is often illustrated by the rubber hand illusion [Tsakiris2005]. An illusion where a subject places his arms on a table, one of his arm is hidden and replaced by a rubber arm. The subject sees the rubber hand being touch by a feather while his real arm is also touched by a feather. After a certain time, if the rubber arm is threaten, the subject demonstrates the same physiological and physical reactions as if his real arm was threaten.
The same experiment has been done in virtual reality with a virtual arm [Sanchez-Vives2010, Yuan2010]. Moreover, it has also been demonstrated on the whole body in an experiment where male subjects see themselves in first person view in the body of a female character [Slater2010] and in third person view throughout the usage of a virtual mirror [González-Franco2010]. The protocols used in the first study is similar to that of the rubber hand illusion. In the case of the mirror experiment, synchronous and asynchronous response of the mirrored avatar to the subject action are used as a stimulus for the body ownership illusion to operate. In both case, the virtual body is then threatened to induce a reaction.
In these experiment, ownership is assessed using the same method as for other type of presence, namely throughout physiological measures, questionnaires, and interviews. The illusion of body or limbs ownership in virtual reality is used, for example, in the treatment of chronic pain [Hoﬀman2000] and for phantom limbs treatment [Murray2007, Mercier2009].
Simulation of Virtual Humans
The most frequently used method of teaching in sports is teaching by demonstration [Desmurget2006]. That method implies a human model on which the learner take the necessary information to build his own representation of an action or gesture. The information the learner retrieves are mostly visual information obtained by the observation of the model combined with audio infor-mation provided by the discourse and the sounds (breathing, clapping, hitting, sounds due to displacement, etc. . . .) accompanying the demonstration. In the case of a virtual learning envi-ronment the model is a virtual human. The level of fidelity of the rendering of that information is therefore crucial. However, the actual technologies do not permit to model all the variables implied in human actions. Simplifications and choices must consequently be made with the risk of degrading and even altering the rendering of the information required by the learner.
Table of contents :
1 State of the Art
1.1 The Usage of Virtual Reality in Sports Teaching and Training
1.1.1 A Brief History of Virtual Reality: From Simulators to Virtual Environments
22.214.171.124 Virtual Reality Definition
126.96.36.199.1 Technological Definition
188.8.131.52.2 Conceptual Definition
1.1.2 Immersion and Presence Factors in Virtual Environments
184.108.40.206 Presence Factors
220.127.116.11.1 External Factors
18.104.22.168.2 Internal Factors
22.214.171.124 Presence Measure Methods
126.96.36.199.1 Objective Measures
188.8.131.52.2 Subjective Measures
184.108.40.206 Presence and Performance
220.127.116.11 Body Ownership
1.1.3 Simulation of Virtual Humans
18.104.22.168 Human Body Representation
22.214.171.124 Morphological Adaptation
126.96.36.199 Motion Representation
188.8.131.52.1 Frame By Frame Representation
184.108.40.206.2 Key-Framed Based Representation
220.127.116.11.3 Signal Processing
18.104.22.168 Motion Production
22.214.171.124.1 Motion Capture Systems
126.96.36.199.2 Motion Capture Data Treatment
188.8.131.52 Section Conclusions
1.1.4 Interacting with Virtual Humans
184.108.40.206 Perception of Virtual Humans
1.1.5 Sports in Virtual Reality: A Study Tool
1.1.6 Teaching and Training Motor Skills in Virtual Environments
220.127.116.11 Video Games
18.104.22.168 Exertion Interfaces
22.214.171.124 Enactive Training Accelerators
126.96.36.199 Teaching by demonstration Using Virtual Humans
2 General Methodology
2.1 Building a Virtual Environment to Evaluate the Relevance of Using Virtual Human for Teaching Motor Skills
2.1.1 The Choice of the Gestures
188.8.131.52 Karate: A Martial Art
184.108.40.206 Three Isolated Basic Gestures
220.127.116.11.1 Tsuki: A Frontal Punch
18.104.22.168.2 Mae geri: A Frontal Kick
22.214.171.124.3 Soto uke: A Frontal Defense
2.1.2 The Design of the lesson
126.96.36.199 The Choice of the Teaching Method
188.8.131.52 The Different Moments of the Lesson
184.108.40.206.1 Video Warmup
2.1.3 The Design of the Virtual Environment
220.127.116.11 Virtual Dojo
18.104.22.168 Virtual Trainer
22.214.171.124.1 Graphical Design
126.96.36.199.2 Motion Capture
188.8.131.52.3 Motion Retargeting on the Virtual Instructor .
184.108.40.206.4 The Virtual Instructor Voice
220.127.116.11 Technological Environment
18.104.22.168.1 Projected Virtual Environment
22.214.171.124.2 Artificial Stereoscopic Vision
126.96.36.199.3 Head Tracking For Egocentric Vision
2.1.4 Studies Participants’ Profiles
3 Teaching with Virtual Humans: A Comparative Study of Teaching in Traditional, Video, and Virtual Environments
3.2.1 Three Different Learning Environments
188.8.131.52 Learning in a Traditional Class with an Instructor
184.108.40.206 Learning from a Video of the Instructor
220.127.116.11 Learning from a Virtual Human Representation of an Instructor
3.2.2 Evaluation by an Expert
18.104.22.168 Initial Evaluation
22.214.171.124 Final Evaluation
4 Teaching with Virtual Humans: Dealing with A Self Representation in the Virtual Environment
4.2.1 Mirror Feedback
126.96.36.199 Real-Time Full Body Motion Tracking and Mapping
4.2.2 New Evaluation Methods
188.8.131.52 Refinement of the Expert’s Evaluation Grid
184.108.40.206.1 Multi-Evaluators Comparison
220.127.116.11.2 Validity of the Grading Scale
18.104.22.168 Evocation Interview
22.214.171.124.1 Interviews Methodology
4.3.1 Evaluation by Experts Results
126.96.36.199 Mae Geri
188.8.131.52 Soto Uke
184.108.40.206 Comparison with Experts’ Scores
4.3.2 Evocation Interview Results
220.127.116.11 Without Mirror Group Results
18.104.22.168.1 First Training Interview Synthesis
22.214.171.124.2 Third Training Interview Synthesis
126.96.36.199.3 Individual Learning Summary
188.8.131.52.4 Group Conclusions
184.108.40.206 Mirror Group Results
220.127.116.11.1 First Training Interview Synthesis
18.104.22.168.2 Third Training Interview Synthesis
22.214.171.124.3 Individual Learning Summary
126.96.36.199.4 Group Conclusions
188.8.131.52 Interviews Conclusions
184.108.40.206.1 The Role of the Indicators of the Learning Environment
220.127.116.11.2 Participants’ Learning Process Management .
5 Standardization of the Performance Evaluation: The Necessity for an Automatic Evaluator
18.104.22.168 Motion Capture of the Evaluations
22.214.171.124 Motion Retargeting of the Evaluations
126.96.36.199 Motion Segmentation in Trials
188.8.131.52 The Choice of the Instructor Reference Trials
184.108.40.206 Automatic Evaluation Based on Biomechanical Parameters .
220.127.116.11.1 Dynamic Time Warping
18.104.22.168 Statistical Analysis of the Distance Matrix
22.214.171.124.1 Distance from the Instructor Reference Trials .
126.96.36.199.2 Intra-subject Variability
188.8.131.52.3 Intra-subject Symmetry
184.108.40.206 Testing of the Automatic Method Evaluator
220.127.116.11.1 Distance From a Panel of Experts
18.104.22.168.2 Distance Between Two Captures of the Instructor .
22.214.171.124 Mae Geri
126.96.36.199 Soto Uke
188.8.131.52 Comparison with the Evaluation of Expert Level Subjects
184.108.40.206.1 Distance Between Two Captures of the Instructor .
6 General Discussion
6.1 Summary of the Studies
6.2 Future Work: Toward a Virtual Coach
6.2.1 Toward an Automatic Evaluator
6.2.2 Toward a more Plausible Virtual Learning Environment
Conclusions and Perspectives
List of Figures
List of Tables
7.1 First Study Evaluation Grids
7.1.1 Tsuki Evaluation Grid in French
7.1.2 Tsuki Evaluation Grid in English
7.1.3 Mae Geri Evaluation Grid in French
7.1.4 Mae Geri Evaluation Grid in English
7.1.5 Soto Uke Evaluation Grid in French
7.1.6 Soto Uke Evaluation Grid in English
7.2 Second Study Expert’s Evaluation Grids
7.2.1 Tsuki Evaluation Grid in French
7.2.2 Tsuki Evaluation Grid in English
7.2.3 Mae Geri Evaluation Grid in French
7.2.4 Mae Geri Evaluation Grid in English
7.2.5 Soto Uke Evaluation Grid in French
7.2.6 Soto Uke Evaluation Grid in English
7.3 Second Study Evaluation by Experts Results
7.3.1 Tsuki Evaluation by Experts Results
7.3.2 Mae Geri Evaluations by Experts Results
7.3.3 Soto Uke Evaluations by Experts Results
7.4 Second Study Evocation Interview Results
7.4.1 Without Mirror Group
220.127.116.11 First Training Session
18.104.22.168 Second Training Session
7.4.2 Mirror Group
22.214.171.124 First Training Session
126.96.36.199 Second Training Session
7.5 Thesis Related Publications
7.6 Other Publications