E-learning: Web-Based Learning

Get Complete Project Material File(s) Now! »

Chapter 5: Usability and Learning: Heuristic Evaluation Criteria

Introduction

This chapter integrates the concepts discussed in Chapters 2 to 4 (Learning theories, Web-based learning and Usability, respectively) of this study in order to derive criteria appropriate for usability evaluation of web-based e-learning applications, and thus to answer the first research subquestion in Section 1.2.2, of this study, namely: • “What criteria are appropriate for usability evaluation of a web-based learning environment for a 3rd year Information Systems course in a tertiary education institution?” The aim is to integrate e-learning and usability. These criteria/heuristics form the basis for the heuristic evaluation by experts and the survey and interview evaluations by learners.
As stated in Section 1.1, evaluations of any educational software should investigate both its pedagogical effectiveness and its usability. Furthermore, in identifying criteria for evaluating elearning applications, the peculiarity of e-learning must be considered, since its primary goal is to support learners in learning the didactic content material in such a way that they devote minimum effort to the actual interaction with the application (Ardito et al, 2004:190) followed by concerted effort in engaging with the learning content. The integration of usability and learning should therefore address usability features that are important to achieve educational goals (Squires & Preece, 1996:15). The evaluation of e-learning applications is further complicated by the increase in diversity of learners, technological advancements and the constantly changing tasks in the real world (Zaharias, 2006:1571).
General interface evaluation criteria, such as Nielsen’s heuristics, given in Section 4.9.1.1, can be applied to evaluate e-learning applications. However, problems arise from applying a small set of principles/heuristics to a wide range of systems (Ardito et al, 2006:272). Specific guidelines are needed in order to evaluate usability of e-learning systems. This study can be viewed as a response to that call, as it proposes an integrated set of evaluation criteria which address both usability and educational aspects of e-learning applications.
This chapter starts with an examination of the rationale for, and the debate around, integration of learning and usability. This is done to highlight some of the issues surrounding the process. This is followed by an overview of existing guidelines and heuristics for usability evaluation of general e-learning applications and educational software. Further, heuristics are identified that are appropriate for the evaluation specifically of web-based e-learning applications.

Rationale for Integration of Usability and Learning

As computers became more powerful and prevalent in the 1980s, educational computing increased and it was realised that the usability of such systems was highly important. Attention was increasingly paid to the role of the end user. Research in the field of human computer interaction began to explore computer usability and the notion of user-centred design (UCD). Now in the 21st century, the HCI community is facing a new challenge of how computers should be used to support individuals or groups of individuals to develop expertise in required skills or in their professions, and in developing deeper and richer understanding of content and practice. A new challenge for HCI is how to support designers who are developing expertise in new and unknown work practices (Quintana, Carra, Krajcik & Soloway, 2002:605). E-learning is one of the areas offering such a challenge, and hence the need arises to integrate usability, from the field of HCI, and learning, emanating from the educational domain.
There is a danger that usability features can be considered at the expense of educational issues. The converse of this is equally true. In some cases there is no consideration of the implication of usability features for educational applications (Jones et al, 1999:499-500). On the other hand, HCI professionals seek to ensure that learners, as users, can interact with technology in a manner that is effective, efficient, and satisfying. While not concerned with learning per se, HCI practitioners seek to include usability features in application systems so as to devise information technology that can be used appropriately by the intended users. However, in the case of educational applications this does not guarantee that learning will occur from using such systems. It will only ensure that willing users can interact with the application in an efficient, effective and satisfying manner (Dillon & Zhu, 1997:223). There are also some arguments that, in general, usability of an educational application in terms of its design does not necessarily result in deep learning. Whether or not this is the case, there is a need to find out how best usability features can be integrated with principled educational design (Mayes & Fowler, 1999:484; Squires, 1999:464). While there are ongoing discussions on the relationship between usability and learning, and acknowledging certain objections to the relevance of usability in instructional software, a study carried out (Tselios et al, 2001:373) shows that there is a positive correlation between usability of the system and performance of the learners in the tasks undertaken.
The main challenge for learners interacting with educational applications, is that they have to deal with two learning processes at the same time: on the one hand, they learn how to interact with the system, and on the other hand, learn how to acquire new skills and expertise. These two are not independent, for they use the same cognitive resources. Therefore, an interface which is difficult to use, could jeopardise the educational success of the system. Whether educational multimedia, in general, do or do not improve learning, studies have indicated convincingly that a difficult-to-use interface negatively affects learning performance (Parlangeli et al, 1999:38). Similarly, a study by Pan, Gay, Saylor, Hembrooke and Henderson (2004:189) concluded that good usability is a necessary condition for learning, for if there is poor usability then the learners waste time attempting to use the system, instead of concentrating on what is to be learned. It was also found that effective learning results in a positive subjective experience.
For most professions, a considerable number of guidelines and heuristics are available. Although much information on usability, information design and graphic arts is available, there is little information on how to create interfaces specific to educational instructional purposes (Lohr, 2000:45). This chapter attempts to suggest criteria that are appropriate for evaluation of web-based e-learning applications, since usability evaluation is an important part of the overall evaluation of any web-based learning environment (Silius & Tervakari, 2003).

Usability and Learning Evaluation Guidelines

This section discusses guidelines that should be considered in the evaluation of e-learning applications in terms of both their usability and pedagogy. As stated in Section 1.1 and 5.2, these dual aspects should be considered in evaluating any educational software. In line with this study, the section concentrates on general principles (criteria) or heuristics for evaluation of elearning applications. Although some new issues that pertain to the intersection of usability and learning in the domains of human computer interaction and education, respectively, are introduced, most of them have been discussed in the previous chapters.
The first two subsections discuss general guidelines for integration of usability and learning. The third subsection gives some of the reasons for using heuristics, instead of checklists, for usability evaluation of educational software. This is followed by an overview of a study done by Squires and Preece to determine appropriate heuristics for evaluation of educational software. This overview also serves as an example of how heuristics for learning are derived. The last subsection provides further heuristics from various other studies.

Human computer interaction and instructional design

The field of HCI comprises the study of the interaction between humans and computers with the general aim of informing the designer regarding technology that is acceptable to human end user. Since the field is multidisciplinary, as discussed in Section 4.1, there is no single theory that guides HCI studies. From the HCI perspective, a number of issues emerge in the design of educational applications. These include:
• The tool;
• The learner;
• The task ; and
• The environment. In particular, focus satisfying – the three main aspects of usability identified in Section 4.3.1. While not concerned with learning per se, HCI practitioners seek to devise information technology that can be used appropriately by the intended users. In terms of multimedia instruction, this would mean including features such as screen readability, adequate support for navigation and task-relevant commands. This would go some way towards ensuring that, in terms of basic human factors, the minimum usability requirements are met. However, as mentioned in Section 5.2, this in itself is insufficient to guarantee that learning will occur from using the system. It can only ensure that willing users will experience a high level of usability in their interaction with the application. Beyond this, HCI turns to instructional design for principles regarding how to take usable technology and apply instructional theory to its pedagogic use (Dillon & Zhu, 1997:222-3).
Learning cannot be approached as a conventional task that entails certain problems to be solved and various outputs to be produced. Learning, by contrast, is a by-product of something else. Learning and instructional systems are not conventional data processing systems. Educational software requires the design of effective tasks, rather than interfaces. The design must be consistent with the tasks at hand (Squires, 1999:465). The design of instructional materials should take into account the following teaching and learning approaches, if it is to be effective (Squires & Preece, 1999:471):
• Constructive approaches;
• Co-operative learning groups;
• Collaboration with other learners via a network or the Internet;
• Strategies to encourage multiple intelligences and a variety of learning styles and independent investigation;
• Open-ended questioning; and
• Strategies to stimulate learners’ creativity.
These issues have already been discussed in Chapters 2 and 3, especially with regard to their relationship to current learning theories and certain e-learning models. The next section addresses the challenges faced in integrating usability with the constructivist learning theory, which is the current dominant intellectual trend in the design of e-learning (Vat, 2001:326)

Usability and constructivism

Constructivist approaches, as discussed in Chapter 2, have been extended to include social perspectives such as situated learning. This implies that the impact of using ICT in learning will depend on the context, with all the components of the learning environment, such as people and artefacts, interacting and contributing to the learning process. It is argued that learning is best supported when technology augments learning rather than attempting to supplant it, and that collaborative learning in which peer group discussion and work is done, is effective in helping students to learn (Squires, 1999:464).
The combination of the principles of constructivism and situated learning is referred to as socioconstructivism. The socio-constructivist view of learning poses challenges for conventional views of usability. Conventionally, usability is conceived in terms of interface design that facilitates efficient and effective completion of well-defined relatively limited tasks. However, socio-constructivism implies that learning environments should be context-sensitive complex environments in which users are expected to make and rectify mistakes as part of the learning process. The need for context-sensitivity is also emphasised in HCI principles for the contextsensitive design and evaluation of software. However there is a mismatch, in dealing with context-sensitivity, between the HCI and the constructivist approach to learning. As already stated, in Section 5.2, usability of an educational application in terms of its design may not necessarily lead to deep learning. There is a need to find out how usability features can be integrated with principled educational design (Squires, 1999:464, Mayes & Fowler, 1999:484). Along similar lines, Squires and Preece (1999:476) distinguish between peripheral usability errors and cognitive errors. Systems should be designed to prevent the former, but to permit the latter while promoting strategies by which learners can recognise them and recover from them.

 Why use heuristics for usability evaluation of e-learning applications?

As discussed in Section 4.9.1, heuristics are general principles that can be used to guide a design decision or critique decisions that have already been made. It is easier for educators and instructional designers to evaluate systems by using heuristic criteria than by using lengthy specific checklists, since the former take less time (Brinck & Wood, 2002:410). Moreover, the use of checklists has been questioned by a number of researchers (Squires & Preece, 1996:15). For example, Rohn, Spool, Ektare, Koyani, Muller and Redish (2002:895) used over 400 guidelines in a checklist-based usability evaluation study. However, as an example, in the context of website design, the heuristic ‘Be consistent’ can be used as a substitute for all the following specific principles (Brinck & Wood, 2002:410):
• Page layouts are consistent throughout the website;
• Page titles are consistent with page names;
• All headers have consistent syntax, capitalisation, and punctuation;
• All bullets are of the same style;
• All logos conform to the same corporate standard;
• Link colour do not vary from page to page; and
• Links colours are consistent with Web conventions: blue for none visited and green or purple for visited ones.
This could reduce the guidelines in the checklists to a few heuristics.
Furthermore, Squires and Preece (1999:471) caution against the use of checklists in the evaluation of educational software due to the following factors:
• It is difficult to indicate relative weightings for questions;
• Checklists focus on technical aspects rather than educational issues;
• Different subjects require different sets of selection criteria; and
• Checklists are tedious to use.
One of the main aims of this study is to establish a set of guidelines that can be used in evaluating WBL applications. The next section is a discussion on some of the major studies that have been undertaken towards the fulfilment of this aim. The discussion takes into account both usability and learning.

Heuristics for learning with software

According to Squires and Preece (1999:473), there is a relationship between the usability heuristics, such as those by Nielsen, Section 4.9.1.2, and the constructivist view of learning, Section 2.2.
The notions of cognitive and contextual authenticity in social-constructivism are used as a basis for relating Nielsen’s usability heuristics to socio-constructivist criteria for learning. Cognitive authenticity, according to constructivists, refers to learning experiences in which learners are helped to construct and refine concepts in personally meaningful ways (Squires & Preece, 1999:469). The three most important concepts that support cognitive authenticity are: credibility, complexity and ownership. Learners will consider a system to be credible if they can explore the behaviours of the system and get meaningful feedback from it. Learners also need help in order to be able to cope with complexities. Learners should be given a sense of ownership of their learning, by allowing them to take responsibility for their individual learning. Contextual authenticity refers to the notion that learning takes place according to the context in which the learning process is situated. The components of the learning environment such as people, including educators and peers, and the artefacts within the environment will all affect the learning process (Squires & Preece, 1999:470). Collaborative learning and the curriculum itself are the most important aspects of contextualised learning. In collaborative learning students learn by peer-group discussion and work. Curriculum refers to the content to be covered by the students.
Table 5.1 shows some of the possible relationships between Nielsen’s heuristics, first column, and the social-constructivist learning criteria, second row. The table shows an interaction between at least one of the heuristics and each of the concepts of credibility, complexity and ownership, which suggests that there is a relationship between cognitive authenticity and usability. There is also some interaction between some of heuristics and the concepts of collaboration and curriculum, which implies a relationship between contextual authenticity and usability. The relationships are, in general, based on the learning theory discussed in Chapter 2, and usability of software applications, as discussed in Chapter 4, thus integrating the two major domains of usability and learning.

Table of Contents
Chapter 1: Introduction 
1.1 Problem Statement
1.2 Research Question
1.3 Value of the Study
1.4 Scope of the Study
1.5 Resources Consulted
1.6 Research Design and Methodology
1.7 Structure of the Study
Chapter 2: Learning Theories 
2.1 Introduction
2.2 Learning Theories
2.3 Conclusion
Chapter 3: E-learning: Web-Based Learning
3.1 Introduction
3.2 Instructional Design and E-learning
3.3 E-learning Support Models
3.4 Design of Web-Based Learning
3.5  Practical Development of Web-Based Learning Sites
3.6  Benefits and Challenges of Web-Based Learning
3.7 Conclusion
Chapter 4: Usability Evaluation: Principles and Methods
4.1 Introduction
4.2  Human Computer Interaction
4.3  Usability
4.4  Usability Evaluation and Classification of UEMs
4.5 Empirical Evaluation
4.6 Model-Based Evaluation Methods
4.7 Observational Methods
4.8 Query Techniques
4.9  Expert Evaluation Methods
4.10  Comparison of Usability Evaluation Methods
4.11  Conclusion
Chapter 5: Usability and Learning: Heuristic Evaluation Criteria 
5.1 Introduction
5.2  Rationale for Integration of Usability and Learning
5.3 Usability and Learning Evaluation Guidelines
5.4 Synthesis of Evaluation Criteria for Web-Based Learning
5.5  Conclusion
Chapter 6: Web-Based Application: Development, Structure and Usage
6.1 Introduction
6.2 The Development Tool and Integration Level of the Application
6.3 Aspects to Consider in Web-Based Learning Development
6.4 Design and Structure of Info3Net
6.5 System Usage
6.6  Conclusion
Chapter 7: Evaluation of Application:Survey and Heuristic Evaluation 
7.1 Introduction
7.2 Survey Evaluation among Learners
7.3 Heuristic Evaluations by Experts
7.4 Comparison of Survey and Heuristic Evaluation Results
7.5 Summary of Main Findings and Chapter Conclusion
Chapter 8: Research Summary and Further Research 
8.1  Introduction
8.2 What has been Achieved
8.3  Answers to the Research Questions
8.4  Further Research
8.5  Conclusion
References

GET THE COMPLETE PROJECT
USABILITY EVALUATION OF A WEB-BASED E-LEARNING APPLICATION: A STUDY OF TWO EVALUATION METHODS