Project topics and materials
Get Complete Project Material File(s) Now! »
Discussion and concluding remarks
Nowadays, increasingly more courses take place online and incorporate rich authentic whole-tasks that provide opportunities for complex learning (van Merriënboer & Sluijsmans, 2009). A research-based instructional design model that has proven to be effective in promoting complex learning (Lim et al. 2009; Melo & Mirando; Sarfo & Elen, 2005), and can be integrated into online learning environments (Frerejean et al., 2019a; Melo & Miranda, 2014), is the 4C/ID-model (van Merriënboer et al., 2002). Nonetheless, offering an online learning environment based on the 4C/ID-model is no guarantee for its effectiveness (Elen, 2020). As the learner is perceived as an active agent in accomplishing learning processes, the effectiveness of online learning environments largely depends on students’ cognitive and motivational-affective characteristics (e.g. Elen, 2020; Jiang et al., 2010; Martens et al., 2004). In order to investigate factors that can improve the effectiveness of online courses, the current PhD project was divided into respectively research track 1 and 2. Research track 1 investigated the influence of students’ cognitive and motivational-affective characteristics on the effectiveness of a 4C/ID-based online course. Former research indicated that the effect of individual differences can be monitored by aligning the learning environment with students’ learning needs (Clarebout & Elen, 2007; Moos & Azevedo, 2009). In view of aligning the online course with students’ cognitive needs, and given the focus on complex learning, research track 2 explored methods for investigating and measuring cognitive load during the problem-solving process by using multimodal data (including physiological data).
As the research project is divided into two research tracks that have largely different research aims, these research tracks will first be discussed separately. In the first part, the main findings of the three studies from research track 1 are explained, combined with the limitations and directions for future research and, theoretical and practical implications. In the second part, research track 2 is approached similarly. Finally, general conclusions are formulated across the two research tracks. The research aim of research track 1 was to investigate the influence of students’ cognitive and motivational-affective characteristics on students’ use of 4C/ID-based online courses and subsequently their learning outcomes. As such, this section will first elaborate on the impact of students’ cognitive characteristics on the use of the learning environment, next on the influence of students’ motivational-affective characteristics on that use, and finally the influence of use on students’ learning outcomes.
Influence of cognitive characteristics on use
In the studies of research track 1, two online courses were systematically designed according to the 4C/ID-model in view of promoting complex learning (van Merriënboer et al., 2002; van Merriënboer & Kirschner, 2018). However, as the learners are active agents in their learning process, the effectiveness of online courses largely depends on the learners’ characteristics (Liem et al, 2008; Martens et al., 2007). Students’ characteristics are even more important in an online context where a lot of autonomy is required from the learners (Tsai, 2013). Autonomy in online courses can largely be manipulated by the amount of learner control that is provided (Väljataga & Laanpere, 2010). In the two online courses that were developed for the studies in research track 1, the components of the 4C/ID-model were offered in a non-embedded manner. Consequently, a large amount of learner control was provided which enabled learners to self-direct their learning. More particularly, students were able to choose their own learning trajectory and select components in line with their personal preferences and cognitive needs (Opfermann, Scheiter, Gerjets, & Schmeck, 2013; van Merriënboer & Sluijsmans, 2009).
In order to investigate whether students selected the components in line with their cognitive needs, Study 3 investigated the influence of students’ prior knowledge on the differences in use of the four components of a 4C/ID-based online course. Results revealed that there was a negative relationship between students’ prior knowledge and part-task practice. More particularly, the lower students’ prior knowledge was, the more students consulted part-task practice. Part-task practice provided opportunities for additional practice of routine subskills of the learning tasks. As a result, these findings seem to suggest that learners with lower prior knowledge realized they did not have the required level of routine subskills (van Merriënboer et al., 2002). By contrast, students with lower prior knowledge did not significantly make more use of supportive information, although this was advisable according to the 4C/ID-guidelines. More particularly, supportive information allows for constructing cognitive schemas in long-term memory which may be absent from students with lower domain-specific prior knowledge (van Merriënboer & Kirschner, 2018). Therefore, findings indicate that differences in students’ cognitive characteristics impact a different use of the components of the 4C/ID-model. Whereas, the larger use of part-task practice is in line with our expectations (i.e. based on the guidelines of the 4C/ID-model), this is not the case for the use of supportive information. When students do not engage as expected and/or do not adequately use the provided support, this phenomenon can be perceived by the instructional designer as ‘instructional disobedience’ (Elen, 2020). Although we do not claim that students in study 3 were ‘instructional disobedient’, our findings suggest that students chose to consult part-task-practice to remediate their knowledge gaps instead of supportive information. Possibly, students felt they mainly lacked the routine subskills of the learning tasks (i.e. covered in part-task practice) instead of the non-routine subskills (i.e. covered in supportive information) (van Merriënboer & Kirschner, 2018).
Influence of motivational-affective characteristics on use
Research track 1 also investigated the influence students’ motivational-affective characteristics on the quantity and quality of use (i.e. Study 1,2) and differences in use (i.e. Study 3). Based on the Expectancy-Value Theory, self-efficacy and task value are assumed to influence academic engagement (Wigfield & Eccles, 2002). As such, in Study 3, the influence of task value and self-efficacy on differences in use of the four components of the 4C/ID-model was investigated. In terms of task value, results revealed a positive relationship between students’ task value and the consultation of learning tasks and supportive information. In Study 3, task value identified students’ interest in the subject matter and how much the student valued the desired outcome (Duncan & McKeachie, 2005). More particularly, this implies that students that were intrinsically motivated put more effort into solving the learning tasks (Chen & Jang, 2010). Moreover, students’ task value influenced the consultation of supportive information. As supportive information is known for connecting present knowledge with novel knowledge, this seems to imply that students with higher task value were more eager to learn new things. This finding is also in line with previous studies showing that students with greater interest, consciously explore additional learning materials (Bong, 2001; Martens et al., 2004). In terms of self-efficacy, despite our expectations, we did not observe an influence of self-efficacy on differences in the use of the four components. Self- efficacious students believe they are capable of executing the learning tasks (Zimmerman, 2000). Consequently, although self-efficacy is believed to be an important characteristic for influencing students’ engagement we did not find empirical evidence for this assumption (Taipjutorus et al., 2012). Nevertheless, it is possible that the effect of self-efficacy in Study 3, might have been outweighed in the research model by students’ prior knowledge as these constructs were moderate related (Kline, 2013).
Findings of Study 3 emphasize the importance of students’ prior knowledge and motivation for grasping learning opportunities. In addition, former research also emphasized the importance of technology acceptance (Šumak et al., 2011). In order to retrieve information on the technology acceptance of the 4C/ID-based online course, PU and PEOU were adopted from TAM (Davis, 1989) in Study 1 and 2. According to TAM, PU is defined as the extent to which learners believe that the online course is a useful learning tool for enhancing their performance. Whereas, PEOU is defined as the extent to which learners believe that using the online course does not entail extra mental effort. Study 1 examined how technology acceptance influenced students’ quantity of use (i.e. total time spent) of the 4C/ID-based online learning environment for teaching French as a foreign language. The study was carried out within the teacher training program for primary school education in which learning how to teach French was part of the students’ training program. Findings suggested that students’ PU of the online learning environment is related to the quantity of use (i.e. time spent). The results correspond with a former study of Juarez Collazo et al. (2012) that also indicated that PU positively influences the use of the online learning environment. Regarding PEOU, findings of Study 1 indicated that PEOU did not influence the actual use of the online course. This might indicate that user-friendliness is subordinate to perceiving the online learning environment as a useful learning tool for learning the subject matter. The second study investigated the relationship between technology acceptance and the quantity of use (i.e. total amount of course activity) and quality of use (i.e. course performance) of an online learning environment for learning French as a foreign language. The results of Study 2 revealed that there was no significant relationship between both PU and PEOU on the quantity and quality of use. The mixed findings of Study 1 and 2 might be related to the fact that the subject matter in Study 2 was not integrated into the students’ training program. These findings emphasize the importance of authentic learning designs (i.e. social and physical context in which it will be used). Former research and educational theory already indicated that authentic learning designs have the potential to improve students’ engagement (Brown, Collins & Duguid, 1989; Herrington et al., 2003; Martens et al., 2004; van Merriënboer & Kirschner, 2018). So despite the use of authentic tasks, learning French may have been less relevant to the students’ everyday lives in Study 2 when compared with Study 1. In the second study, TAM was expanded with an external variable, namely, perceived instructional quality. In order to retrieve information on students’ perceived instructional quality, students were asked to fill in the Teaching and Learning Questionnaire (TALQ) developed by Frick et al. (2009). This questionnaire incorporates the five First Principles of Instruction of Merrill (2002). Merrill’s (2002) First Principles of Instruction define that learning is promoted when learners are engaged in solving real-world problems, existing knowledge is activated, new knowledge is demonstrated, applied, and integrated into the learners’ world. Findings of Study 2 indicated that the perceived instructional quality had a significant positive influence on students’ PU and PEOU (i.e. technology acceptance). These findings seem to indicate that technology acceptance and the perceived instructional quality of the online course are largely interrelated. This relates to former studies that plea for the extension of the TAM model with pedagogical quality (e.g. Lee et al., 2009; Liaw & Huang, 2013; Yang et al., 2017).
Additionally, Study 2 investigated the relationship of technology acceptance and perceived instructional quality on the quality of use (i.e. course performance). Findings indicated no association between technology acceptance and course performance, whereas students’ perceived instructional quality was related to course performance. This latter finding indicates that learners who recognized the pedagogical quality invested more mental effort into solving the exercises qualitatively. Findings can be related to the study of Frick et al. (2010) who indicated that integrating the First Principles of Instruction of Merrill (2002) in the course design is related to higher course performance. However, it is not excluded that students who had a greater knowledge of French, also had a better understanding of the relevance of the learning activities, and therefore were more motivated to attain pre-established goals (Elen, 2020).
Influence of use on learning outcomes
In Study 1 and 3 the influence of respectively the quantity of use and differences in use of the 4C/ID-based learning environment on students’ learning outcomes was investigated. Findings of Study 1 indicated that the quantity of use of the online course, influenced students’ learning outcomes when controlling for students’ prior knowledge. Additionally, Study 3 indicated that differences in use also improved students’ learning outcomes, when controlling for students’ cognitive and motivational characteristics. More specifically, the combination of the activity on learning tasks and the consultation of procedural information seemed to improve students’ learning outcomes. The procedural information provided instructional guidance which helped learners to perform the routine subskills of learning tasks. Procedural information was offered just-in-time while solving the learning task and as such improved learning (van Merriënboer & Kirschner, 2018). Based on the Cognitive Load Theory (CLT) this might indicate that the provision of instructional guidance prevented learners from focusing on irrelevant aspects. As such, the students were able to devote more working memory resources to learning (Sweller, 2010; van Merriënboer & Kirschner, 2018). Nevertheless, we must also point out that given the high relationship between prior knowledge and students’ learning outcomes, there must have been little learning gain. This might indicate that students mainly solved learning tasks they already mastered. This may also partly explain why there is no relationship between supportive information (i.e. includes new subject matter), part-task practice (i.e. practicing routine subskills) and learning outcomes. Students probably invested little effort in learning and practicing new subject matter. This might have been the result of the fact that the subject matter of Study 3 was not part of the students’ training program (Brown et al., 1989).
Table of contents :
1. Introduction
1.1. Complex learning
1.1.1. Defining complex learning
1.1.2. Challenges for complex learning related to students characteristics
Cognitive characteristics
Motivational- affective characteristics
1.1.3. Designing learning environments for complex learning
1.2. Online learning environments for complex learning
1.2.1. Digitalization in educational settings
1.2.2. Shaping online education
1.2.3. A 4C/ID-based online learning environment
1.2.4. Technology acceptance of an online learning environment
1.2.5. The use of Learning Analytics to investigate (online) learning processes
1.3. Research aims and overview of the conducted studies
1.3.1. Research track 1
1.3.2. Studies of research track 1
1.3.3. Research track 2
1.3.4. Studies of research track 2
1.3.5. Overview of the doctoral thesis
2. Technology acceptance of a 4C/ID- based online course
2.1. Introduction
2.2. Theoretical background
2.3. Method
2.4. Results
2.5. Discussion
2.6. Conclusion
3. The perceived instructional quality of a 4C/ID-based online course
3.1. Introduction
3.2. Theoretical background
3.3. Method
3.4. Results
3.5. Conclusion
4. The influence of cognitive and motivational characteristics on differences in use
4.1. Introduction
4.2. Theoretical background
4.3. Method
4.4. Results
4.5. Discussion
4.6. Conclusion
5. Combining physiological data and subjective measurements
5.1. Introduction
5.2. Theoretical background
5.3. Method
5.4. Results
5.5. Discussion
5.6. Conclusion
6. Physiological data: a promising avenue to detect cognitive (over)load?
6.1. Introduction
6.2. Theoretical background
6.3. Method
6.4. Results
6.5. Discussion
6.9. Conclusion
7. Discussion and concluding remarks
Research Track 1: individual differences determining the effectiveness of 4C/ID-based online courses
7.1. Main findings of Research Track 1
7.1.1. Influence of cognitive characteristics on use
7.1.2. Influence of motivational-affective characteristics on use
7.1.3. Influence of use on learning outcomes
7.2. Limitations and future directions
7.3. Implications of research track 1
Research track 2: MMLA for measuring cognitive load during online complex problem-solving
7.4. Transition of research track 1 to research track 2
7.5. Main findings of Research Track 2
7.5.1. Self-reported cognitive load versus physiological data
7.5.2. Physiological data for assessing cognitive load
7.6. Limitations and future directions
7.7. Implications of research track 2
In sum: let’s keep it complex
References
