Development of quality audit criteria for chronic disease management

Get Complete Project Material File(s) Now! »

Development of Quality Audit Criteria for Chronic Disease Management 

The purpose of this chapter is to provide an overview of the quality audit criteria development process as relevant to this thesis and present the criteria that have been developed. The chapter first provides some background information related to some initial work that laid the foundation for the work presented in the rest of the thesis. This is followed by a discussion on the criteria development process and details of a validation exercise that was carried out to ensure the validity of the audit criteria. The chapter concludes with a discussion on several high level categories of audit criteria that have been identified to provide the maximum reasonable level of abstraction fromthe developed criteria.

Background – the Therapeutic State Transition Model

I started my PhD research around the time a series of meetings were being held (refer to Section 3.2 for details) between The University of Auckland based researchers and an expert panel from a general practice the researchers were collaborating with. The requirements of the expert panel included quality indicators such as ‘patients who have been on concurrent therapy with angiotensin-converting enzyme inhibitors (ACEi)/angiotensin receptor blockers (ARBs) and diuretics’ and ‘patients with lapse in antihypertensive medication’, where a lapse in antihypertensive therapy is said to commence when all antihypertensive medications, if taken as directed from the day of prescribing, run out (brief lapses are expected and may not be problematic where the patient has retained some prior supply). I was involved with the technical implementation of these criteria, and to satisfy the expert panel audit requirements such as medication lapses, I used the therapeutic state transition model set forth by my PhD supervisor, Prof. Jim Warren and others [12] and then further enhanced by Gadzhanova et al. [10]. Within the therapeutic state transition model, CPG recommendations are first abstracted into key therapeutic states. EMR prescribing data is then mapped into the corresponding states which are then used to identify cases on suboptimal care by interpreting the transitions in patient states with respect to the CPG recommendations [12]. In essence, this model looks at analysing chronic disease prescribing actions over time in terms of transitions in status of therapy as indicated in the EMR prescribing records [10].
According to the therapeutic state transition model, each prescription produces two events – one marking the start of the prescription and another implicit event marking the expected end of the prescription if directions (dose, frequency and repeats/refills for example) given by the GP were properly adhered to. By using various state variables to denote antihypertensive therapy (this was the primary domain of interest – see Section 3.2) as shown in Table 3.1, a series of therapeutic state transitions were constructed which are effectively points in time when the status of key aspects of a patient’s therapy changes (for example, being on ACEi/ARBs – i.e., state ‘A’ in Table 3.1 and running out of medication; i.e., moving into a Zero state). This model is further enhanced by heuristically processing the states to avoid over-sensitivity with lapses (indicated by Zerostate) of less than 90 days being coalesced into the prior state, as is any other state of less than 30 days duration. My implementation of the therapeutic state transition model as relevant to this thesis is described in [21]. Also included in the analysis are combination products that represent more than one of the above groups – ATC: C07BA, C07BB, C03AB, C08GA01, C09BA, C09BB and C09DA. Based on the therapeutic states in Table 3.1, a state transition overview diagram (Figure 3.1) can then be constructed for the practice, and this process is described in [10, 12]. This diagram presents a patient count of various state transition movements within the particular practice during the defined time period. The arcs in the state transition overview diagram indicate the number of patients within the practice with that particular transition. This view of the practice does not show how an individual patient has moved through different transitions, and to cater for this requirement, individual path diagrams have been proposed [11].
For a hypothetical patient, Table 3.2 shows the state variables for the prescriptions the patient received during a selected time period together with a brief explanation on the state transitions that occur. Figure 3.2 shows the corresponding individual path diagram for this patient.

Development of Audit Criteria and The Quality Audit Report

Around the time I started my PhD, my supervisor was putting together a University based research team to collaborate with a general practice in West Auckland with a predominance of Pacific Island patients. I became a member of this team which conducted a series of meetings with the collaborating general practice. The practice formulated an expert panel for this work consisting of five members; two GPs, the practice manager and two nursing staff. During the first meeting the opportunity was given to the expert panel to propose suitable criteria for auditing their management of patients with chronic disease, focussing mainly on hypertension and its common comorbidities. This was the background to the audit criteria development initiative which will be described in some detail in the remainder of this section. This research was approved under the University of Auckland Human Participants Ethics Committee protocol number 2007/078.
After the initial meeting, three one-hour meetings with the expert panel were conducted on the premises of the practice between May 2007 and July 2007 to develop a quality audit report based on EMR data from the practice. The information needs for audit were determined primarily by the practice’s panel, while the University based researchers also provided their input (the University based research team had two medical doctors).
The quality audit report was designed to document:  Descriptive data about the practice (e.g., prevalence of hypertension);  Positive attributes (numbers that the panel would like to raise – e.g., percent of patients diagnosed with hypertension with BPnow controlled)  Quality improvement opportunities / cautionary attributes (cases registering in this category are recommended for review)
After each of the three meetings, queries were implemented to populate the quality audit report from practice data, based on data extracted from the practice’s commercial PMS (MedTech32 [226]). The protocol was to extract prescribing and relevant laboratory data and observations for the 18-month period preceding the first meeting, with the exception of problem diagnoses (based on Read Codes [227]) which are relevant for an indefinite time with respect to chronic disease and hence were extracted for as far back as five-years. Use of unique practice identification codes for each patient (internally, MedTech32 refers to this identifier as ‘MMID’ which will be used throughout this thesis to indicate a unique patient identifier) allowed the practice to identify patients while maintaining patient anonymity in data used by the external investigators. Note that this MMID was not the NZ National Health Index (NHI) number (which is a unique patient identifier used within the NZ health system for clinical and administrative purposes), but a practice and PMS specific identifier. Also, within MedTech32, diagnoses are referred to as ‘classifications’, and therefore these two terms are used interchangeably in this thesis.
Although the extracted data span was 18-months (with the exception of classifications which was for five-years), the reporting window (i.e., the evaluation period) was narrower than this as shown in Figure 3.3. For this particular report, an evaluation period of 12-months was suggested by the expert panel. The reason for extracting prescriptions and other relevant data for 18months (with the exception of classifications) was to develop the gradual build-up of therapy using therapeutic state transitions. This was required due to the nature of chronic diseasewhich usually spans over an indefinite period of time, hence will be inaccurate to say that a hypertensive patient was not on antihypertensive therapy just because there was no antihypertensive prescription at the very beginning of the reporting period (as therapy that started sometime before the beginning of the evaluation period could have continued into the evaluation period). This period (six-months in this case) prior to the beginning of the evaluation period is referred to as the run-in periodfrom here on.Relevant laboratory tests were identified in advance of the first meeting by analysis of guidelines (notably, JNC7), but extended based on meeting results (in particular, to include estimated glomerular filtration rate; eGFR). The queries required some pre-processing to compute the duration of medication supply for each prescription as indicated by dose, frequency, pack size and repeats (generally 90-days). Lapses were identified where a medication, if first dispensed on the day of prescribing and subsequently taken as directed, would have run out. Periods of lapse were computed both for overall antihypertensives and for several antihypertensive drug groupings of interest to the panel, including ACEi and ARB (collectively), beta-blockers and thiazide diuretics. The computation of lapse periods was based on the therapeutic state transition model discussed previously (Section 3.1) with appropriate settings for the heuristics (e.g., lapses less than 90 days were not coalesced into the prior state as shorter lapses needed to be identified).
The audit criteria developed during the meetings with the expert panel were put together in the form of a quality audit report containing three main sections – a descriptive section (with 7 criteria) with general information about the practice, a section with supportive indicators (with 12 criteria) which contains information the practice in general would want to drive to a 100 percent and a final section (with 17 criteria) which contains cautionary indicators. Table 3.3 shows two examples from each section of the quality audit report, one indicating a summative or an outcome related indicatorand the other relating to a process or a treatment. The supportive and the cautionary sections of the quality audit report contain quality indicators under several subcategories. The supportive section constitutes of: BP related, prescription related, effective combination therapy, continuity of therapy and drug-problem indications; the cautionary section has the subcategories: generic (for indicators such as ‘patients classified with hypertension and not prescribed with antihypertensive agents within 90 days of classification’), BP related, lack of continuity of therapy, drug-drug interactions, drug-problem interactions and monitoring. An instance of an actual quality audit report that was generated with the criteria together with resulting patient numbers for this particular practice can be found in Appendix I. Some important temporal issues that need to be considered when formulating the quality indicators are discussed in Chapter 5 where temporal requirements have been identified methodically.

An Initial Implementation of the Audit Criteria

Data Extraction, Data Cleaning and Pre-Processing

The first phase of the implementation of the developed criteria was visiting the practice and extracting the required data from the practice’s commercial PMS (i.e., from MedTech32) to a password-protected Microsoft (MS) Access research database. MedTech32 is widely used by NZ GPs to manage their patients; however, being a commercial PMS system, MedTech32 does not provide direct access to its internal database structure. Although the underlying database tables cannot be accessed directly, the system provides a database querying interface via a visual query builder tool that lets an interested user extract certain database fields (that includes everything we needed for our purposes) and save this information into plain text or commaseparated values (CSV) files. The query builder tool was used to extract the required data, however, a disadvantage was that if the database fields were of type memo (used mainly for lengthy fields such as clinical notes, and laboratory test results), the resultant files would include many control characters often used for formatting. Therefore, after the data was extracted, a data cleaning phase was required to filter out the control characters and extract the required values. Visual Basic for Applications (VBA) modules are part of the MS Access database management system suite with direct access to the data tables in the database, and therefore cleaning was performed using VBA modules.
BP measurements also needed some cleaning and pre-processing. Within the PMS, there are two ways of entering BP values so that they get stored in the database table that maintains BPs; one is to enter the values directly into the BP fields provided in the software as an observation (of type ‘BP’), and the other is to enter the values into the physician notes field as plain text with a backslash followed by the letters bp or BP (i.e., \bp or \BP). However, there were many instances where BP values were entered into the notes field without following this convention; for example, “BP – rt arm sitting 190/90” and “rpt Bp 150/85 at 2.15 pm, felt better…”. All such instances of BP measurements were extracted from the notes using VBA regular expressions and added to the BP table in the research database. Further, some BPs were correctly entered into the field provided within the PMS as well as entered into the notes field; in such cases only one record was included.
After the data was cleaned and prepared to be processed for analysis, the prescription durations were determined using the PMS’s computed duration field if it was available. If the duration field was not set, the duration was computed based on instructions (signatura) given by the GP consisting of dose, frequency, number of repeats (refills) and the quantity. In NZ, a prescription (usually issued with two refills) is valid for a period of three months before it expires (medication may be dispensed as 90-days supply at once, or may be collected one month at a time from a single 90-day prescription). However, during visual inspections of the data several anomalies related to ways of recording prescription durations were identified (see Figure 3.4).
The usual method of computing prescription duration is (1+RPT)*(QTY/Consumption) where consumption is determined based on the SIGS (i.e., the instructions). So for example, if the SIGS is “1 tabs, Once Daily”, consumption would be 1, and if QTY=90 and RPT=2, then duration would be 270. However, in NZ a prescription duration cannot be 270 days. Therefore cases in Figure 3.4 (i) need to be interpreted as cases where GP intention was to prescribe a total of 90 tablets over 90 days as indicated in the PER field and computing duration based on repeats, quantity and consumption will be incorrect. The reason for this inconsistency is because the PMS system usually automatically calculates the PER field based on other fields (repeats, quantity and built-in SIGS), but when PER values are entered manually/overridden (e.g., when non built-in SIGS are specified), this field may not get automatically populated/updated. Cases (ii) and (iii) in Figure 3.4 show cases where RPT=0/2 and QTY=30, however, the PER field is set to 30 and 90 respectively. Due to such anomalies, heuristics were used to populate a new ‘Duration’ field in the database instead of using PER as the duration of a prescription. This Duration field was set to 90 if PER was 90 (irrespective of RPT/QTY) or else it was computed using quantity, repeats and consumption.
The nature of chronic illness is such that once a patient is diagnosed with a chronic condition such as hypertension or diabetes, it becomes a life-long condition. Usually it is expected that patients get classified with a chronic condition within the PMS only once, and although it is indeed the case with majority of the patients, non-scientific analyses based on ad hoc SQL queries show that approximately 40% of all hypertensive patients get classified with hypertension more than once. In fact, in one extreme case, a particular patient in the patient cohort was classified with Hypertensive disease (based on Read Clinical Codes) a surprising 21 times within the space of only five years (Figure 3.5).
It is unclear why such entries for a single chronic condition for the same patient get entered into the PMS multiple times by the same physician, but it could be due to a range of factors including physicians not going through the patient history to look at any previous/existing classifications (possibly due to consultation time constraints). Whatever the reason may be, it is an existing issue and when such cases were encountered, the earliest classification date was used as the classification date for that patient for that particular condition simply due to the nature of chronic illness.
The raw data extracted from the practice were imported into the database with table names having the format raw_Type, where Type refers to prescriptions, classifications, BPs, demographics and labs. After the data was imported into the database and cleaned accordingly, several new intermediate database tables were created from the raw tables to facilitate the querying process (which was essentially the data preparation stage). These tables, such as those with updated prescription durations were named im_Type. Several other tables were also created to store the required knowledge for queries; notably: – separate tables to hold classification information: ReadCodes_HT (to hold the hypertension related Read Codes), ReadCodes_Diab (to hold the diabetes related Read Codes) and so on depending on the classification requirements of the quality audit report. – a table to hold the different drug-drug class information based on therapeutic state variables (see Table 3.1for the different drug classes).

Table of Contents
Abstract
Acknowledgements
Table of Contents
Lists of Tables
Lists of Figures
Glossary of Terms
1.0 INTRODUCTION
1.1 The Burden of Chronic Disease
1.2 The Opportunity
1.3 The Problem
1.4 The Research Question
1.5 The Proposed Solution
1.6 Thesis Outline
2.0    RELATED WORK
2.1 Computer Interpretable Clinical Practice Guideline Modelling Techniques
2.2 Guideline based Chronic Disease Management CDSSs
2.3 Temporal Analysis and Visualisation
2.4 Quality Indicators in Primary Care
3.0   DEVELOPMENT OF QUALITY AUDIT CRITERIA FOR CHRONIC DISEASE MANAGEMENT
3.1 Background –the Therapeutic State Transition Model
3.2 Development of Audit Criteria and The Quality Audit Report
3.3 An Initial Implementation of the AuditCriteria
3.3.1 Data Extraction, Data Cleaning and Pre-Processing
3.4 Criteria Validation –The 20/20 Study
3.5 Four Categories of Quality Audit Criteria
4.0   PRELIMINARY RESULTS –AN ONTOLOGY BASED APPROACH
4.1 Limitations of Using an Entirely SQL-based Approach (the Previous Implementation)
4.2 Domain-Modelling and Methods
4.3 Identifying Patients with Hypertension on Suboptimal Therapy using the Ontology Based Approach
4.4 Other Ontology Related Work and Limitations of a Pure Ontology Based Approach
5.0    CHRONOMEDIT FRAMEWORK ARCHITECTURE
5.1 The Need for a Novel Criteria Model
5.2 Framework Verification
5.3 Introducing Medication Adherence as an Audit Criterion
5.4 The Criteria Model
5.5 The Computational Framework
6.0    FRAMEWORK TESTING
6.1 An Overview of Software Testing Concepts
6.2 Methods
6.3 Identification of Test Cases
6.4 Random Testing to Ensure Software Reliability
6.5 Errors Detected
6.6 Limitations of Framework Testing
7.0    APPLICATIONS USING CHRONOMEDIT
7.1 Data Extract
7.2 Identifying patients satisfying the eight audit criteria
7.3 Investigating ACEi/ARB adherence issues and the relationship to BP control in patients with hypertension and diabetes
7.4 Antihypertensive adherence and impact on BP control
7.5 Interval Based Measures as Quality Indicators in Blood Pressure Management
7.6 Comparison to two indicators from the Quality and Outcomes Framework to Enhance Quality of Pay-for-Performance Incentives
7.7 Comparing dispensing based adherence to prescribing based adherence
7.8 Improving GP awareness of antidepressant adherence issues
7.9 A feasibility study of nurse-led adherence promotion
8.0    DISCUSSION
8.1 Significance
8.2 Related Work
8.2.1 Frameworks and systems
8.3 Scope/Limitations
8.4 Future directions
Appendix I –A Quality Audit Report
Thesis Related Publications
References

GET THE COMPLETE PROJECT
ChronoMedIt –A Computational Quality Audit Framework for Better Management of Patients with Chronic Disease