Project topics and materials
Get Complete Project Material File(s) Now! »
Seeking evidence to support usability principles for medication-related CDS functions
Introduction
Medication computerized Clinical Decision Support (CDS) functions have been shown to have a positive impact on patient safety by improving prescribing practices [1] and reducing Adverse Drug Events [2]. However, they remain difficult to implement and face acceptance problems [3]. Moreover, they may also generate technology induced errors, i.e., latent types of errors related to the usage of this technology [4]. The root causes of such problems are usually of Human Factors (HF) nature and more specifically related to usability features. However, the evidence of the relationship between the ob-served/reported outcomes, the usage problems experienced by the users, the usability flaws involved in the usage problems and the usability principle whose violation led to the usability flaws remains unsubstantiated most of the time.
This paper presents a four-stage framework describing the chain of consequences and inferences linking usability principles to clinical outcomes. The consequences of the framework in terms of method and interpretive expertise are discussed and applied to the design of a systematic review of usability evaluation studies of medication related CDS functions.
Background
Figure 2 describes the proposed four-stage framework putting together usability principles, usability flaws, usage problems and outcomes in the work system. It can be applied to any kind of Healthcare Information Technology (HIT) system. Two processes connect the four stages. The top-down process describes the propagation of usability flaws until they finally impact the healthcare system’s performance through deteriorated outcomes, e.g., medical errors. The bottom-up process describes the research, evaluation and expert consensus process which allows (i) identifying and characterizing actual usage problems and (ii) infer from them elements of evidence to support usability principles for corresponding HIT systems.
Usability flaws of technical systems such as Computerized Prescriber Order Entry (CPOE), CDS functions or medical devices result from violations of usability principles in the design of those systems. Most of those usability flaws create usage problems when the system is put into use. The importance of the usage problems experienced by the users depends on several variables (e.g., the nature of the usability feature violated or the type of task supported by the faulty function). Ultimately, these usage problems may actually negatively impact the healthcare system performance, e.g., by slowing down the clinical workflows or generating medical errors characterized as technology-induced errors [4]. Again, the scope and importance of the negative outcomes depend on several variables, e.g., the severity of the usage problems but also characteristics of the context of use and of the socio-technical organization in which the system has been implemented along with the capacity of adaptation ingrained in the work system. The bottom-up process aims at linking the outcomes identified by evaluation and impact studies back to usage problems and their usability root causes (usability flaws) and ultimately to the corresponding usability principles that have not been complied with. Given the multiplicity of other HF or technical factors that are intertwined with usability variables in the described top-down process, this inference work is far from trivial and requires a sound expertise in HF and usability of HIT applications. Moreover, most of impact studies such as clinical trials of CPOE or CDS functions lack qualitative analysis that would allow identifying the usage patterns of the HIT system acting as intermediate variables explaining the observed outcomes. Qualitative HIT evaluation studies may prove more informative but still require HF expertise in the analysis of work systems to differentiate organizational vs. usability issues, given that the report of the evaluation study provides enough details to make this inference. Usability studies aiming specifically at identifying usage problems often fail linking the observed usage problems with causing usability flaws [5]. Such studies require usability expertise in HIT systems to properly infer the usability flaws. Finally, usability studies listing usability flaws of evaluated systems do not always provide the necessary level of details (e.g., screenshots) to establish clear links with violated usability principles and descriptions of potential related usage problems.
Figure 2. Emergence and potential propagation (dotted lines) of usability flaws to healthcare outcomes and seeking evidence for usability principles.
Usability principles are the core part of the framework. They can be divided into two categories: (1) methodological principles to apply the user centered design or the usability engineering process [6, 7] and (2) usability principles and features of the targeted products [8, 9]. This paper deals only with the latter category. The most important part of those principles is reported in standards elaborated on the basis of international expert consensus. They may be relatively general (e.g., usability principles for the design of Graphical User Interface (GUI) [10] that would apply to all HIT applications) or they may be more specific to a category of product (e.g., medical devices of a certain kind). Unfortunately, most standards face several problems that prevent usability principles from an easy and unambiguous interpretation by non-experts [11].
During the last decade, there have been several initiatives to identify the most important usability principles for medication CDS systems. These attempts are mostly based on the experience of the authors in the domain and on lessons learned from medication CDS functions design and implementation projects in which they participated [12-15], or derived from a specific theoretical approach of the cognitive processes involved in the interaction of the users with medication CDS functions [16].
Rationale
The great variety of usability principles, whether recommended by standards or scientific publications, and their lack of comprehensible organization prevent developers, and even HF experts, from identifying those they should apply, and therefore from applying them completely and correctly. Moreover, the lack of evidence to support usability principles may also lead systems’ developers to question the legitimacy of the stated principles. We aim at seeking evidence supporting the organization and prioritizing of those usability principles.
The systematic review method is a technique allowing the emergence of evidences from HIT evaluations’ published reports. As far as we know, it has been used only once in the field of medical management systems to find evidence of usability flaws in CPOE systems [17]. In order to seek evidence to support usability principles for medication CDS functions, we designed a systematic review based on the four-stage framework. This review proposes to answer two main questions:
• What features are characterizing medication CDSS usability?
• Do those features generate usage problems of the CDS function and ultimately outcomes in the work system?
Applying this framework allows identifying precisely the inferences necessary to jump from one stage of the framework to another. It also supports the search query through the delimitation of the scope of relevant evaluation studies, the definition of inclusion/exclusion criteria and the design of the interpretive grid for the analysis of final set of papers.
Systematic review design and process
The design of the systematic review follows as far as possible good practice recommendations [18-20]. The key concepts involved in the review, “medication CDS functions” and “usability”, have been defined in the light of the framework. This supports the latter definition of inclusion or exclusion criteria of the papers.
Concepts definition
Usability
Usability is defined as “the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specific context of use” [21]. Usability goes way beyond the features of the GUI (e.g., legibility of the texts, layout and prompting of information and tools), and deals more generally with the fitting between the system behavior and its users’ needs. Considering medication CDS functions, one of the most important usability features which violations may engender important usage problems and negative outcomes is the compatibility of the system with clinicians’ activities, essentially of cognitive nature [16]. For the review (mostly the papers analysis phase) we consider four dimensions in the usability concepts:
• HCI characteristics;
• How the system responds to users’ actions;
• Organization or accuracy of the knowledge incorporated;
• Availability of functions required to support users’ tasks, especially of cognitive nature.
Therefore, the analysis of retrieved papers, both for decision upon inclusion or exclusion and for final systematic analysis, requires a deep HF knowledge of intended users’ needs, activities and working procedures.
Medication-related CDS functions
CDS functions refer to a very wide range of tools: documentation forms-templates, relevant data presentation, order-prescription creation facilitators (e.g., order sets), protocol or pathway support, reference information and guidance, alerts and reminders (pushed or pulled) [16].
As the review focuses on HIT tools, medical devices in which medication CDS functions have been integrated are excluded from the analysis (e.g., auto-injectors pens, pumps). Only medication CDS software used in hospital or general practice in the internal medicine field is considered. To increase results’ homogeneity, we focus on alerting systems (alerts and reminders). As a result, the review includes studies of software supporting the management of e-prescriptions by physicians, pharmacists and nurses. Thus, CDS functions integrated in Bar Coded Medication Administration and e-Medication Administration Records are excluded from the review.
The literature was searched from 1980 to 2012 using PubMed, Scopus and Ergonomics Abstracts databases. Two semantic groups of key terms were constructed with the support of medical terminologies experts: terms related to CDS functions, alerts and CPOE and terms related to HF (cf. Table 1). As far as possible, MeSH terms were chosen for PubMed database thanks to the Health Multi-Terminology Portal
[22]. The key terms have been slightly adapted for Scopus and Ergonomics Abstracts databases. Key terms in each group were combined with the operator “OR”. Then both groups were combined using the operator “AND”. Queries have been performed in March 2012 and updated on the 26th October 2012. The search retrieved a total of 5862 items.
Table of contents :
General Introduction
Chapter 1. Seeking evidence to support usability principles for medication-related CDS functions
Chapter 2. Usability flaws of medication-related alerting functions: a systematic review
Chapter 3. Impact of usability flaws in medication alerting systems on usage and work system
Chapter 4. Usability design principles for medication alerting systems: literature synthesis and study of their coverage regarding related usability flaws
Chapter 5. Methods uncovering usability issues in medication-related alerting functions: results from a systematic review
Chapter 6. Perceived usefulness of a usability issues reporting form to help understand “usability-induced use-errors”: a preliminary study
Discussion and Conclusion
APPENDICES
Appendix 1. French summary
Appendix 2. Curriculum Vitae and Portfolio
Appendix 3. Supplementary material
