Immersive Technologies (XR) in Assembly 

Get Complete Project Material File(s) Now! »

Immersion and XR

Oxford dictionary defines immersion as “deep mental and social involvement in something”. Witmer and Singer (1998) describe it as a subjective experience within an interactive environment. Slater (2003) simply explains immersion as the degree of technology delivery in all sensory modalities and tracking capabilities to their equivalent in the real world. McMahan (2003) adds that immersion results from the user’s cognitive captivation in the virtual world, while Brown and Cairns (2004) add that total immersion means a total loss of awareness of the real world. Jennett et al. (2008) clarifies that the involvement in the task, which causes a lack of attentiveness for time and space as well as of a sense of being in the task environment. There is a huge potential for immersive technologies in the digitalization age, especially in the services sector. According to Pallot et al. (2013a) the immersion concept can take physical, cognitive or collective form according to the purpose and feature. Immersive prototyping is an innovative prototyping process that utilizes technologies to immerse the stakeholders for a specific purpose. Conferring to Dupont et al. (2016) where it immersion is considered as the perception of being physically there while in immersive reality providing the ability to interact and communicate with immersive environments, where one or more of the five senses are engaged.. Moreira et al. (2013) explained that immersive prototyping requires an accurate configuration of targeted immersive environment as such that it is both technically and functionally similar to the real one, with strong focus on specific stakeholders’ needs and objectives.
According to Pallot et al. (2013a) immersive technology forms vary from VR, AR, and MR. Dupont et al. (2016) add that these immersive technologies could be utilized to fool the eyesight, hearing and haptic; where a total immersion is when the five senses are observing the immersive reality as actual including the natural intuitive interactions. Barnes (2017) explains that by moving around or using an immersive device equipped with the sensors and trackers to gather information to be able to alter and adjust the immersion. Cummings and Bailenson (2015) state that immersive technologies are becoming more affordable. Sutherland (1968) was the first to define a graphics-driven Head Mounted Display (HMD), as his idea was to combine it with tracking devices. information. Rolland and Hua (2005) distinguished HMDs by their user perspective, either by being monocular, bicular, or binocular. Powerwall is considered as a large high resolution display screen, so users can move, navigate and view freely an immersive environment (Ball et al. 2007). The first powerwall was built in 1994 in the University of Minnesota, USA. The first Cave Automatic Virtual Environment (CAVE), which is typically a spatial display within a larger space, was invented (Cruz-Neira et al., 2012). Cruz-Neira et al. (2012) established the distinction between passive immersion, as a user is seated and having a 360° scene and wearing a HMD and active immersion, where the user is freely moving in 360° in a scene in a CAVE. Fast-Berglund et al. (2018) defines eXtended Reality (XR) as a term that contains all real-and-virtual combined environments, including user interactions with wearables like VR, AR, and MR.
According to Thon (2008) the idea of attention alteration is vital to the concept of immersion. Lombard et al. (2009) states that immersion is understood as a user’s mental state when they are involved, or engaged. Nechvatal (2009) suggests that the more the technology is pushed forward the more the immersive the experience will be better. Per Pallot et al. (2013), an immersive platform is an assembly of both hardware and software with a specific 3D content application, where VR, AR and MR technologies are established as different immersive platforms. Immersion is the main concept that is responsible for the added value of immersive technologies. Nordin et al. (2014) describes immersion as “real-world dissociation”. Agarwal et al. (2019) defines immersion as the state of deep mental engrossment, usually experiencing awareness and surrounding. Pallot and Richir (2016) defined “immersiveness” as “the state of being deeply engaged, recognized as a tactical and sensory-motoric immersion; or fully absorbed in solving a problem, seen as a strategic and cognitive immersion; or reading a captivating story or watching an exciting movie, considered as a narrative or emotional immersion”. Immersion concept can take physical, mental or social form depending on the purpose and attributes (Pallot et al. 2013). Conferring to Pallot and Richir (2016) on existing immersive platforms used in various applications address senses by immersing one or more of the five human senses, this is represented in form of an immersive User eXperience (Pallot and Richir, 2016, Dupont et al., 2017, Pallot et al., 2017). Pallot et al. (2016; 2017) and Dupont et al. (2016; 2017) also add that immersive technologies are used to fool the human senses; whereas total immersion constitutes the five senses to perceive the immersive reality as real, while allowing natural intuitive interactions.

Immersion and Cybersickness

Many researchers theorize that the immersive experience be contingent to the user engagement degree (Agarwal and Karahanna, 2000; Brown and Cairns, 2004). Pallot et al. (2013) suggests that an immersive experience hinges on the degree of the user’s immersion and presence, engagement and enjoyment. Jannett (2009) suggests that Real World Dissociation (RWD) measures the sense of time, awareness of surroundings and mental responsiveness, and Nordin et al. (2014) proposed that immersion is RWD, while Dupont et al. (2017) add that the three RWD properties depend on the extent to which one or more persons are absorbed by a common task. Pallot et al. (2013) argue that it depends on the feeling when equipment, time and surrounding disappear from the user’s mind. Dupont et al. (2016) suggest that immersive technologies usually fool three senses through visual, auditory and tactile channels. There are demonstrated benefits in using immersive services prototypes, like exploring and learning with a VR prototype, which is learning by doing without any risk (Pallot and Richir, 2015). Krueger (2011) and Rizzo et al. (2015) have demonstrated that Immersive Virtual Environment (IVE) can be utilized for therapy or other post-traumatic rehabilitation. Cummings and Bailenson (2015) claim that the higher the quality of immersion, the higher the sense of presence in the IVE. According to Wirth et al. (2007) and Cummings and Bailenson (2015), presence comprises of user sense of self-location and interaction potential with the IVE. However, there are also drawbacks to using immersive virtual environments; according to Lawson (2014), it can induce motion sickness, vertigo, dizziness, visual tiredness and nausea. According to Pallot and Richir (2015), the three key challenges for realizing IVEs are: (a) improving immersion quality; (b) increasing easiness of immersion creation; (c) reducing the risk of the forms of sickness in immersive environments. Furthermore, when using HMDs instead of a costly power-wall or CAVE, the resulting immersive experience and side effects might not necessarily be the same due to the blindness of the physical environment (Pallot and Richir, 2015). Dupont et al. (2016) mentions that full immersion occurs when the five human senses observe the computer-generated reality as physical reality.
Davis et al. (2012) mention that cybersickness, motion sickness, and simulator sickness have comparable indicators; however, they are instigated by different experiences. Kennedy et al. (1993) argue that there is an association between cybersickness, motion sickness, and simulator sickness in their primary physiological causes and symptoms forms, which might seem to be related yet dissimilar. Stanney et al. (1997) described the difference between cybersickness and simulator sickness, where cybersickness is described as a constellation of indications of uneasiness and sickness induced after a VR experience. LaViola (2000) suggests that cybersickness can result in multiple symptoms, such as: nausea, disorientation, headaches, and eyestrain. Cobb et al (1999) added that 80% of participants that experienced a VR system had some form of cybersickness in the first ten minutes. Chen et al. (2011) suggest that the likelihood of cybersickness in VR systems are around 30%, while Kim et al. (2005) argue that it could be over 80%. Rebenitsch and Owen (2016) mention that while cybersickness has been a recognized issue in VR and AR systems for years, much of the nature of cybersickness as an ailment is still vague. Weech et al. (2019) propose that presence and cybersickness are negatively related. Dennison et al (2016) recommend neurophysiological and non-physiological measures to estimate the change in cybersickness during immersion. Davis et al. (2012) believe that understanding the causes of cybersickness is a necessary step in making virtual environment more useable. According to Richir and Pallot (2017), the main implication of immersion due to the phenomenon of illusion (false 3D) is that it provokes brain and visual tiredness that cause uncomfortable immersive experience.

Virtual Reality

According to the American Heritage Dictionary (2005) “virtual” is defined as the “existing or resulting in essence or effect though not in actual fact, form, or name », which is « created, simulated, or carried on by means of a computer or computer network ». VR classically refers to the application and use of interactive computer generated simulations created to allow users with to engage in environments that appear and feel similar to real world objects and events (Sheridan, 1992; Weiss and Jessel, 1998). VR is composed of interactive computer simulations (Sherman and Craig, 2018) capable of three-dimensional replications that have seemingly tangible bodily user interactions (Dioniso et al., 2013). This transfers the sensory information to a user (Abari et al., 2017) to induce a behavior by using artificial sensory stimulation with little or no consciousness of this interfering (LaValle, 2017). The obvious strength of the VR approach is the method it assimilates users with the virtual environment allowing also the manipulation of virtual objects, and the performing of other actions in a manner that tries to immerse the user completely within the virtual environment. The progression in VR technology has provided the motivation for adopting and applying it efficiently in various different industrial applications such as design, modelling, process simulation, manufacturing planning, training, and testing. (Mujber et al., 2019). Ermi and Mäyrä (2007) suggest that the potential of VR in learning is frequently linked with its power to offer users with the immersion and presences feelings. Hwang and Hu (2013) mentioned that the main VR benefit is its utilization as a tool to improve the understanding of intangible or intricate concepts. Narraro-Haro et al. (2016) reported that VR captures the attention in a manner that improves interference outcomes in experimental contexts. In an experimental learning context, VR emotional component is thought to create a greater impact compared to traditional training (Faria et al., 2016).
Schwald and De Laval (2003) advocated that even though AR is a novel technology than VR, it has been considered and used in several service sectors such as, training and maintenance. Grenfell and Warren (2010) added that immersion is frequently quoted as one of the motivations for using virtual reality for learning. Huang et al. (2010) suggest that immersive experience becomes vital to comprehend unknown concepts of the target user. Vince (2004) mentions that designers could design concepts and explore them at a virtual level long before they were created and that the applications are limitless when we reach the desired technological level. As VR is a well-established technology, there are numerous definitions from diverse research domains. As such a definition list was created to display the main VR characterization and definitions as shown in Table (2.3).

Mixed Reality

MR is not a well-established technology yet, especially that there is still a division in different definitions used by researchers. The basis for MR is the virtuality continuum formulated by Milgram and Kishino (1994) a concept to describe the immersion range from real environments to virtual environments including all the possible mixed forms. They also state that MR covers the area between the two extremes of reality and virtuality by merging the real with virtual worlds. In this current age, the technological advancement in MR is increasing, as such a simple classification that was presented by Milgram and Colquhoun (1999) is no longer adequate. It is necessary to include finer divisions of the different MR (Schnabel et al. 2007). Schnabel (2009) defines a mixed environment as the intersection of real and virtual environments where physical and digital elements co-exist, and interact and amalgamate. Schnabel (2009) also explains that MR integrates digital information seamlessly in the user’s environment allowing the user to interact with these digital objects and information while presenting it together in a single experience unlocking new boundaries of engagement in collaboration. Dunleavy and Dede (2013) clarified that MR interfaces combine real and virtual environments to enable mental immersion in an environment that combines physical artefacts and digital information. Schart and Tschanz (2017) described MR as the hybrid or merged reality, lying between VR and AR, where virtual objects expand the environment as if they were one recognizing the surroundings and displaying digital objects in the area. The effectiveness of MR as a tools for training, especially in manufacturing and maintenance activities is a continuing debate including experts from different fields to attempt to define a mutual framework for designing and assessing MR tools for the industrial applications. On the other hand, what is apparent from the success of the Microsoft Hololens, and the Hololens II in the market and the numerous applications using MR technologies, especially if looking at the project from the past three Laval Virtual Conference, are increasing every year as industrial acceptance and adoption was apparent. There were several definitions for MR, the most relevant are shown in Tab. (2.5).

Conventional Service Prototypes

We consider conventional service prototypes are the prototyping forms that use conventional methods for representing and displaying the prototype. As the CSP as a keyword were not explicitly mentioned in the literature, a search was done on alternative search terms for the literature review. Stark et al. (2009) developed a smart hybrid prototyping approach, which includes different technologies and tools to create prototypes like VR, AR, MR, mockups, sketches and simulation. Exner et al. (2014) compared prototyping methods according to their fidelity levels, differentiating between physical (i.e. paper, physical mockups, sketches), virtual (simulations, digital mockups, MR), and both (i.e. AR, functional mockup, rapid prototyping). Several researchers attempted to differentiate and categorize service prototyping forms; (a) Rudd et al. (1996) with implementation, (b) Walker et al. (2002) with application, (c) Holmquist (2005) differentiated them into low and high fidelity, (d) Kim et al. (2006) with usability, (e) Blackler (2009) with type of interaction, and (f) Blomkvist and Holmlid (2010) suggests that is a variety of approaches and activities. To make it easier to understand the concept of conventional service prototyping, a classification for the forms was established in our previous publication (Abdel Razek et al., 2018a).
These different SP forms are differentiated by their method of application and representation. The conventional SPs are categorized into four forms, (1) Verbal Service Prototypes (VSP), (2) Paper Service Prototypes (PSP), (3) Mock-Up Service Prototypes (MSP), and (4) Simulation Service Prototypes (SSP). All these keywords that represent CSPs didn’t exist in the literature in these forms, still through the literature review we found similar definitions and characterizations that helped in defining these terms in a later sub-section.

Verbal Service Prototypes

Verbal Service Prototypes (VSP) are based on engaging stakeholders by increasing their understanding of a new service idea verbally, there was no definite definition for VSP found in the literature. Bill Moggridge explains in his keynote speech at the Danish Service Design Symposium “when you put all these things together, with elements from architecture, physical design, electronic technology from software, how do you actually prototype an idea for a service, and it seems that really, it’s about storytelling, it’s about narrative” (Moggridge 2008). The verbal communication is key for stakeholders to understand and give feedback. Blomkvist and Holmlid (2011a) explain that VSP aims to uncover the real add value of storytelling when releasing the needs of stakeholders as well as help them realize new opportunities, this all while deepening provider’s understanding of service hotspots. From our understanding of the literature we will consider the VSP definition from our previous publication, as relies on verbal communication to create a cognitive stimulus to engage stakeholders in a narrative or story (Abdel Razek et al., 2018a). VSP could be used with conjunction with other prototyping forms. It could be done as an introductory prototyping process, to start a brainstorming session or to deduce stakeholder’s requirements for example. VSP creation requires little effort, nevertheless a skilled narrator is vital to effectively engage and influence stakeholders. The main indicator of success is the stakeholders’ engagement, acceptance level, and their amount of feedback.

Paper Service Prototypes

According to Ehn and Kyng (1992) paper prototyping is considered as one of the most utilized prototyping form in service design, particularly when aiming on involving stakeholders. Brandt and Grunnet (2000) clarify that paper service prototyping makes the stakeholder’s involvement in the service development process feasible. Synder (2003) considers Paper Service Prototypes (PSP) as a representative variation of user’s usability testing by performing tasks while using paper. Kangas and Kinnuen (2005) suggest that PSP can be used to get key insights on stakeholders’ requirements, and identify stakeholders’ gains and pains without any technological investment. Paper prototyping emphases on co-creation, by gathering information from stakeholders in form of fuzzy paper mock-ups; where several iterations could be used, to improve the prototype. Paper service prototyping could be used as a preliminary prototyping process when prototyping a service. Paper service prototyping involves little to no effort, and its main success indicator is stakeholders’ engagement and the quality of feedback. Paper is a practical prototype form as it is readily available virtually everywhere, and its shelf life is very long if stored properly; on the other hand, it is not the most sustainable material. PSP enables creating a rapid service prototypes by using handwritten notes, and drawings, as it is a skill that virtually every stakeholder masters and can do in a short period of time.

Mock-Up Service Prototypes

The word “mockup” stems from Circa 1915–1920, from the verb mock up by imitation which comes from the French word “maquette” which translates roughly in a prototype (Webster’s New World College Dictionary “mockup”). In an industrial context, a mock-up could be anything from a scale or a regular size depiction to representation of a design, equipment, machine. Mock-ups are also widely accepted as tools for teaching, demonstrating, designing, evaluating, and testing in various industries. Papantoniou et al. (2016) considered a mock-up also as a prototype if it delivers some design functionality for testing. Mock-ups are often used by designers to acquire feedback from stakeholders about design ideas early in the design process. Mockups could be constructed in a physical or a digital form as discussed by Exner et al. (2014). Morris (1992) describes a physical prototype as an early form model used in assessing design, fit, form and performance. Antionino and Zachmann (1998) characterized a digital Mock-up as a realistic computer simulation of a product with the capability of all required functionality covering every process from design, manufacturing, and service. Greasley (2004) adds that mock-ups could be also used to communicate a new service idea. Moritz (2005) explains that rough mock-ups can help stakeholders understand the service idea quickly, while perfect mock-ups help in evaluating and explaining it. Mock-ups can be used to assess the limits and possibilities of service aspects. Fontaine et al. (2009) indicated that a mock-up could communicate the service idea before its creation. According to Miettinen et al. (2012), a mock-up can be realized in using various methods and material, which can be iteratively enhanced to introduce more detailed prototype as the need arises for evaluating more precisely. Blomkvist and Holmlid (2012) add that mock-ups comprise either one physical element or a combination of them that can be made from different materials and by different methods. Physical mock-ups are routinely used for assembly tests (Bernard, 2005).

Table of contents :

1. Introduction 
1.1. Background, Purpose and Motivation
1.1.1. Prototyping
1.1.2. Product Prototyping
1.1.3. Service Prototyping
1.1.4. Thesis Context and Motivation
1.1.5. Purpose
1.2. Objectives, Research Approach and Questions
1.2.1. Objectives
1.2.2. Research Approach and Questions
1.2.3. Research Questions
1.2.4. Hypothesis
1.3. Research Significance
1.4. Thesis Structure
1.5. Summary
2. Literature Review 
2.1. Introduction
2.2. Service
2.3. Immersion and XR
2.3.1. Immersion and Cybersickness
2.3.2. Virtual Reality
2.3.3. Augmented Reality
2.3.4. Mixed Reality
2.4. Prototyping
2.4.1. Service Prototyping
2.4.2. Service Prototype
2.5. Experience
2.6. Immersive Technologies (XR) in Assembly
2.7. Identified Gaps
2.8. Proposed Research Framework
2.8.1. Research Questions and Hypothesis
2.8.2. Proposed Service Prototype eXperience Definition
2.8.3. Proposed SP research Model
2.9. Summary
3. Methods 
3.1. Introduction
3.2. Context
3.3. Research Approach
3.4. Research Methods
3.5. Statistical Analysis
3.6. Experiment Preparation
3.6.1. Testing
3.6.2. Baseline Experiment
3.6.3. SP Experiment
3.6.4. Experiment Mixed Reality Extension
3.7. Industrial Workshop and Focus Group Discussion
3.7.1. Workshop
3.7.2. Focus Group Discussion
3.8. Summary
4. Findings 
4.1. Introduction
4.2. Baseline Experiment
4.2.1. Observations Results
4.2.2. Qualitative Results
4.2.3. Summary
4.3. Model Statistical Validation
4.4. SP Experiment
4.4.1. Paper Service Prototype
4.4.2. Mock-Up Service Prototype
4.4.3. Virtual Reality Service Prototype
4.4.4. Augmented Reality Service Prototype
4.4.5. Mixed Reality Service Prototype
4.6. Industrial Workshop
4.6.1. Focus Group Discussion
4.6.2. Feedback Survey Results
4.6.3. Focus Group Discussion Results
4.7. Summary
5. Discussion 
5.1. Introduction
5.2. Limitations, Reliability, and Validity
5.3. Responding to the Research Questions and Hypothesis
5.4. Outcomes Summary
5.5. Comparing to Recent Studies
5.6. Contribution to the Body of Knowledge
5.7. Contribution to Industrial Practices
6. Conclusion
References 

GET THE COMPLETE PROJECT