Rating the Strength of Recommendation and Quality of Evidence of CPGs 

Get Complete Project Material File(s) Now! »

Limits of guideline compliance

Even if CPGs proved to enhance clinical practice, several causes limit their effectiveness and, consequently, the adherence and compliance of clinicians with CPGs. (Grimshaw and Russell 1994; Davis and Taylor-Vaisey 1997).
The complexity of the medical domain makes the formalization of CPGs a difficult task to be achieved successfully. First, formalizing evidence is not a straight forward task and may not reach the correctness and knowledge definition level that clinicians would expect, since opinion and interpretation still have a huge influence on healthcare management. On one hand, CPG development procedures are quite constraining, considering that guidelines are developed for population healthcare management, assuming that the concept of a “standard” patient exits, but might be inaccurate or even wrong for particular patients in real populations (Hurwitz 1999). The opposite can happen as well, when small randomized clinical trials or controlled observational studies are used to report evidence that may need to be generalized, resulting in poorer outcomes when treating bigger populations (Shekelle et al. 1999). The importance of personalization of the guidelines is imperative in order to improve adherence and compliance rates. The work of (Bouaud and Seroussi 2002) states that for breast cancer management guidelines, 66% out of 127 patients fit correctly to be evaluated with standard guidelines, whereas 39% of the cases suffer a bias between the guidelines recommendation and the treatment administered.
A closely related and important issue is the guideline development process or how CPG development working groups are composed. Usually, these teams are comprised of quality auditors or managers who are guided by their opinions, interests, and experience, and who intend to formalize evidence seeking appropriateness of the provided recommendations but ignore the iterative and causal reasoning of clinicians (Woolf et al. 1999). Depending on the clinical context and according to the approaches followed for developing and disseminating, as well as the applied implementation methods, CPGs can be more or less successful when reporting the latest clinical evidence (Grimshaw and Russell 1993). Even if CPGs are audited to rate their quality of evidence and strength of recommendations, trying to replicate the clinical reasoning process is difficult and translates into simplified, generalized, and in some cases ambiguous vocabulary, which may lack the supporting evidence and will require the clinicians’ own opinions for its interpretation. The act of giving way to interpretation and providing purely clinical judgment-based recommendations can be very susceptible to bias and/or directly non-compliant with CPG-based recommendations and following one´s own self-interests (Shekelle et al. 1999). Defining the followed reasoning process as much as possible would help to track and identify the causes of these evidence gaps and to analyze the reasons behind biases from guidelines.

Rating the Strength of Recommendation and Quality of Evidence of CPGs

CPGs rely on the latest EBM to guide clinicians in the decision-making process. Scales such as the Appraisal of Guidelines Research and Evaluation (AGREE) assess the quality of the guideline’s development process, not focusing on the clinical content and the quality of the evidence of the provided recommendations (AGREE Collaboration 2003). Hence, to what extent are the recommendations provided in the CPGs based on high-quality evidence? What is considered as high-quality evidence? How can clinicians and CPG developers be confident about those recommendations?
In the last decade, several approaches have been developed in an attempt to answer these questions and formalize evidence-grading systems. The Agency for Health Care Quality and Research (AHRQ)4 reviewed the ongoing efforts of different medical groups and reported that there are currently over 100 proposals for grading the evidence of the guideline recommendations (West et al. 2002). Since many of these approaches were complex and difficult to integrate in daily clinical practice, the AHRQ stated three key elements to be covered by any evidence grading system that would facilitate their dissemination throughout the clinical community (Clair 2005): (i) quality, referring to the validity of the study or the minimal opportunity of bias that it could have, (ii) quantity, when talking about the number of studies taken into account to formalize that evidence and the number of subjects studied within them and, (iii) consistency among other studies on the same topic that could be comparable. Some of the approaches that do accomplish these criteria are the Oxford Centre for Evidence-Based Medicine (OCEBM) Levels of Evidence5, the Cochrane Collaboration6, the US Preventive Services Task Force7 (USPSTF), the Strength of Recommendations Taxonomy (SORT) and the Grading of Recommendations, Assessment, Development and Evaluation8 (GRADE). The first five are more focused on reporting evidence based on patient-oriented outcomes, which may disagree with disease-oriented outcomes. For example, when analyzing a disease or condition such as providing Doxazosin for treating hypertension or high blood pressure, a disease-oriented outcome would be that it reduces the patient’s blood pressure to prevent suffering a stroke whereas a patient-oriented outcome reports that this same treatment increases mortality in people of African ancestry. Patient-oriented outcome approaches are more simplistic in order to facilitate their implementation throughout CPGs and are mainly developed for specific clinical domains or illnesses (Ebell et al. 2004a).

Clinical Knowledge Formalization

The formalization of the clinical knowledge contained in CPGs into a computer-interpretable structure is a mandatory step in order to move towards a computerized healthcare system, a standardization of the decisions made over time, and ease the assessment of new evidence and patient outcomes’-based results. The transformation of CPGs into CIGs is a legitimate issue since it is a time and resources consuming task. Moreover, providing some tools to ease their maintenance and updating would be helpful. Retrieving clinical performance information would allow the comparison of the provided clinical care among similar clinical cases based on patients’ outcomes and identify biases from the guideline-based recommendations.

Decisional Event Structure

A decision can be defined as the final conclusion or a resolution reached after analyzing the available information, ideally, based on some validated criteria generated from previous evidence or personal experience. A decision in the clinical domain seeks to provide the best personalized care for the analyzed patient, based on the latest clinical evidence along with the clinicians’ intuition and experience (Bate et al. 2012). In this thesis, decisions were formalized into a digital structure named the Decisional Event (DE) to (i) gather all the information related to the decision-making process, reflecting all the rationality for taking a decision and the consequences of such decision for the patient and (ii) allow its later analysis to retrieve information about CPG compliance rates and new clinical evidence, based on the clinical success reported on different patients’ outcomes. The DE structure is defined by the following set of components:
• P = {Pi }: A set of clinical patient parameters that define the clinical case. This structure is flexible enough to handle any kind of clinical data coming from any type of clinical domain, supporting different kinds of data type values (e.g. Boolean, integer).
• R = {Rj }: A set of clinical statements or rules expressed in a computer-interpretable way as IF-THEN specifications. This formalization originally comes from conditional computer programming, where different statements or Boolean conditions are defined to be accomplished and provide a conclusion or a recommendation. In this case, statements were formalized following the knowledge coming from different clinical sources (e.g. CPGs, local guidelines, experience-based rules generated by the system). Each of the formalized rules consists of the following items:
o A = {Am }: a set of the clinical statements that compose the conditional part of the rule (i.e. the IF-part) a priori defined in CPGs. These clinical statements must be checked by the clinical parameters {Pi } to trigger the consequent part, meaning that they must achieve the mathematical condition imposed on all elements of the set, which in turn, when they are multiple, are connected by logical operators (i.e. and, or). If all conditions are satisfied, the recommendation or THEN-part will be given as a conclusion or recommendation for the studied case. o W: the recommendation(s) defined as the consequence part of the rule (i.e. the THEN-part) describes the conclusion or the recommendation(s) to be followed when the conditional part of the rule was accomplished by the studied clinical case. Each recommendation can be composed of a single order or an ordered list of orders. We identify mainly two kinds of orders: (i) characterization orders, whose objective is to provide or define a value for a clinical parameter depending on the clinical variables studied in the conditional part and (ii) action orders, which will define the clinical or therapeutic action to perform. A recommendation can be composed of several orders of one type or both types. Hence, giving a recommendation could change the value of some clinical variables while also suggesting an action to perform, expressed as 𝑊=(𝑆1,𝑆2,…𝑆𝑙) 𝑤𝑖𝑡ℎ 𝑙>1 where Si is each atomic order.
• FD: the Final Decision represents the decision of the clinical team at a time t0. This final decision is defined as an atomic recommendation from the structural point of view and it can be compliant with one of the recommendations provided by the rules, W, or define a new proposition coming from the clinicians’ experience or new clinical evidence not reported in the guidelines, hence being non-compliant with them.
• E: the Executed Treatment administered at time t1 by the clinical team to the patient after the decision was made, is also defined as an atomic recommendation from the structural point of view and it can be compliant with one of the recommendations provided by the rules W, and the final decision FD, taken at the time t0 or be non-compliant to one or both of them. We measure compliance at these two levels because independently of the compliance of the final decision with the guideline-based recommendations, the final executed treatment may be non-compliant due to new information that was not known at the time of the decision was made. These biases must be recorded since E is the real treatment given to the patient.

READ  Electrolyte thermodynamic model and the State of the art 

Methodology for an evolutive CDSS

Guideline-based CDSS relies on the knowledge reported within the CPGs to provide the best patient-specific and evidence-based recommendations. These systems are developed in order to promote the compliance of the standardized CPGs aiming to reduce the variability of the clinical practice and improve patients’ clinical outcomes. To achieve this objective, CPGs, which are paper-based narrative documents, must be formalized as CIGs.
As stated in Chapter 2.1, a number of proposals have been made for CPG formalization as CIGs. In the proposed approach, the clinical knowledge in CPGs was formally represented as “Task-Network Models” (TNMs) since this formalism allows the representation of the clinical knowledge as Business Rules (i.e. IF-THEN rules), showing the dependencies among the different clinical conditions which, when fulfilled in a satisfactory way, provide some therapeutic recommendations. This CIG approach is implemented in a Java-based formalization of the rules (i.e. Rule object presented in Chapter 3.1.1), allowing computer-based reasoning and execution of these formalized guidelines through a rule-execution engine for a given case study.
In the following, the issues faced when formalizing and updating CIGs within a CDSS will be addressed, providing a domain-independent methodology that facilitates CIG maintenance and update in a computer interpretable format. Moreover, an authoring tool that supports and eases the reviewing and validation process of the formalized CIGs is presented. Finally, an experience-based CDSS is proposed which tracks all decisions made over time generating new experience-based rules to deal with the identified guideline limitations, enriching the knowledge base of the CDSS with experience and exploring some outcomes’ analysis to assess the strength behind these experience-based recommendations.

Domain-independent CIG formalization

In this Chapter, a tool for formalizing any domain CIG, leaning on the previously formalized Java Rule object within the Decisional Event (see Chapter 3.1.1) is proposed. Since the main objective is to generate automatically CIGs with no clinical domain restrictions, a modular approach based on the Drools22 business rule management system was done, which encompasses two main components:
(i) an automatic and domain-independent CIG rule file generator in Drools Rule Language (.drl extension) .
(ii) a regular standalone runtime execution environment or rule engine that will communicate with the first module for triggering the dynamically updated CIGs.
The first module presents a generic approach to ease the CPG formalization into a CIG (e.g. DRL file) in a very flexible and domain-independent way, using Drools Rule Language or DRL-based rule templates like the one shown in Figure 16.

Augmenting the clinical knowledge using experience

As mentioned before, guideline-based CDSSs are sometimes insufficient for providing the best recommendations for particular clinical cases. This could be the case for several reasons, such as (i) outdated CPGs implementation, to which we proposed a modular architecture that could help in overcoming that issue, (ii) particular clinical cases that do not fit with the guidelines due to their complexity or the lack of supporting evidence formalization in the CPGs or (iii) the inclusion of other factors that are closely related to the decision-making process and can alter the compliance with the guideline proposed recommendations, such as clinical preferences or patient preferences. To deal with those gray areas of CPGs, an experience-based CDSS is proposed in this chapter, whose main objectives are (i) the formalization of the clinical experience or implicit knowledge that guides clinicians when they do not comply with the guidelines, using it to generate new experience-based rules that will cover those guideline gaps and would support future similar clinical cases, and (ii) structuring and gathering all the information related to the decision-making process, allowing the measurement and evaluation of the provided healthcare.

Outcomes for usability and strength computation

Outcomes can be used to assess and evaluate the clinical reliability of the knowledge formalized in the CPGs at a time t, supporting clinicians during the decision-making process for the most reliable and best healthcare performance assessment when administering a treatment to the studied patient. In our approach, we propose dynamically storing outcomes in a computerized way to allow for the automatic evaluation and update of the evidence reported by the CPGs. We focused on three kinds of outcomes: (i) first given treatment response, (ii) toxicities or adverse events and (iii) patient-reported outcomes.

Table of contents :

Table of Contents
1. Introduction
1.1 Background
1.2 Problem Analysis Context
1.3 Objectives and Research Questions
1.4 Research Scope
1.5 Approach
1.6 Structure
2. State-of-the-Art
2.1 Clinical Practice Guidelines and their transition to Computer Interpretable Guidelines
2.2 Clinical Decision Support Systems
2.3 Limits of guideline compliance
2.4 Rating the Strength of Recommendation and Quality of Evidence of CPGs
2.5 Visual analytics in healthcare
2.6 Conclusions
3. Research Design Approach
3.1 Research Design Concepts
3.1.1 Clinical Knowledge Formalization
3.1.2 CIG Formalization Module
3.1.3 Ontology-based semantic validation
3.2 Methodology for an evolutive CDSS
3.2.1 Domain-independent CIG formalization
3.2.2 Authoring tool
3.2.3 Augmenting the clinical knowledge using experience
3.3 Visual Analytics
3.3.1 Decisional Events visualization
3.3.2 Real-World Data visualization
4. Use Case: Breast Cancer
4.1 Breast Cancer Knowledge Model (BCKM)
4.2 Breast Cancer Clinical Practice Guidelines
4.3 Non-compliance criteria definition
4.4 Breast cancer outcomes
4.5 Interaction of the guideline-based CDSS and the experience-based CDSS within DESIREE project
5. Validation
5.1 Technical Assessment
5.1.1 Unit test design and implementation
5.1.2 Integration test design and implementation
5.2 Clinical assessment
5.2.1 Experience generation from retrospective data in simulated BUs
5.2.2 Performance and clinical validation with prospective data in real BUs .
6. Conclusions
7. Research contributions
7.1 International Conferences
7.2 Journals
7.3 Awards
7.4 Intellectual Property
8. Discussion and Future research


Related Posts