The effect of evidence-based, treatment-oriented, clinical practice guidelines on improving patient care outcomes: a systematic review protocol

Source:

JBI Evidence Synthesis

June 2016, Volume :14 Number 6 , page 42 - 51 [Free]

Keywords

Clinical practice guideline, evidence-based guideline, patient care, patient care outcomes

Authors

Colbeck, Marc
Lockwood, Craig
Peters, Micah
Fulbrook, Paul
McCabe, Delia

Abstract

Review question: The question that this systematic review aims to address is: does the use of evidence-based, treatment-oriented, clinical practice guidelines by healthcare professionals result in improvements in patient outcomes?

Article Content

Background

Importance of guiding medical practice - why use clinical practice guidelines?

Effectively discovering and integrating data from primary research into a clinical practice is a challenging and labor-intensive endeavor, and physicians tend not to do it very effectively.1 The title of the aptly named 2010 study Seventy-Five Trials and Eleven Systematic Reviews a Day: How Will We Ever Keep Up?2succinctly summarized the challenge of keeping up-to-date with the literature. As long ago as 2001, Shaneyfelt3 declared that a general internist would need to read 20 articles a day, every day of the year, to remain up-to-date with medical literature.

In reference to how well clinicians are succeeding in keeping up-to-date with current research, Grol and Grimshaw reported the following:

"One of the most consistent findings in research of health services is the gap between evidence and practice. Results of studies in the USA and the Netherlands suggest that about 30-40% of patients do not receive care according to present scientific evidence, and about 20-25% of care provided is not needed or is potentially harmful."1^(p.1225)

To address this challenge, professionals in the field of health care in the 1990s began to focus on the development of clinical instructional documents that summarized research for busy clinicians and presented them as a set of generalized guidelines or instructions to bridge the gap between academic research and clinical practice.4

Previous systematic reviews

A review of clinical practice guidelines (CPGs) over 20 years5 showed that although the general quality of CPGs had moderately increased over the time period of the study (published in 2010), the scores for rigor of development, which the authors suggest "may be a stronger indicator of quality than any of the other domains of the instrument"5^(p.3) (referring to the AGREE II instrument) have been quite low (mean of 40% compliance with best practices), and did not improve over the last five years of the study, suggesting that the evidence base of CPGs is not improving or is only improving moderately. Additionally, several other recent studies6,7 have shown that many CPGs are based on low-quality evidence.

Therefore, the concept of CPGs has been enhanced by the recommendation that CPGs should be based on the evidence. Evidence-based clinical practice guidelines (EB-CPGs) are summaries of the best evidence in a field which also include systematically developed recommendations to guide clinicians in their care of identified patient groups, based on that evidence.5 "Evidence-based" means that the CPGs are created on the basis of rigorous, unbiased and transparent methods of collating and appraising information using the best scientific findings of the highest quality and value to assist in providing optimal care to the patient.8

In support of this, a 1998 study9 showed that clinicians were more likely to use and were more likely to support the use of evidence-based CPGs in comparison with non-evidence-based CPGs. Correspondingly, the most recent version of the often-referenced definition of CPGs developed by the Institute of Medicine (IOM) has been updated to include mention of evidential quality, and now reads: "statements that include recommendations intended to optimize patient care that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care options".10^(p.4)

The IOM further clarifies that, To be trustworthy, guidelines should:

* "be based on a systematic review of the existing evidence;

* be developed by a knowledgeable, multidisciplinary panel of experts and representatives from key affected groups;

* consider important patient subgroups and patient preferences, as appropriate;

* be based on an explicit and transparent process that minimizes distortions, biases and conflicts of interest;

* provide a clear explanation of the logical relationships between alternative care options and health outcomes, and provide ratings of both the quality of evidence and the strength of the recommendations; and

* be reconsidered and revised as appropriate when important new evidence warrants modifications of recommendations."10(p.5)

Despite the growing consensus that CPGs should be based on clearly explicated evidence, it is not yet possible to say that all CPGs are reliably evidence based. A recent study11 critically appraising the quality of CPGs used in intensive care found that less than half met a predefined measurement of evidentiary quality.

Since the introduction of the concept of CPGs, and the further refinement of EB-CPGs, there has been a proliferation of clinical practice guidelines. In September 2015, a search of the Guidelines International Network's database resulted in 674 CPGs that specifically dealt with the treatment of conditions.12

Despite the proliferation of CPGs and EB-CPGs, and despite the intuitive logic of such tools, there is little concrete evidence to support the statement that CPGs result in a meaningful improvement in patient health-related outcomes. There have been five previous systematic reviews evaluating the effectiveness of CPGs.13-17

In these reviews, the populations under study varied widely; methods were heterogeneous and in many cases, the studies evaluated by the authors of the systematic reviews were described as scientifically inadequate. Additionally, the measurements in the studies that were evaluated focused on a wide variety of indicators, including health-related systems improvements, health-related process improvements and health-related outcome improvements, as well as patient satisfaction measurements. All the systematic reviews identified state that the amount and quality of evidence upon which to develop their conclusions were very poor and that more high-quality research is needed. Each of the five systematic reviews will be considered below.

A 1993 study by Grimshaw and Russell13 included any randomized control trial, as well as before- and after-studies with controls, but excluded uncontrolled trials. It evaluated improvements in both the process and the outcomes of care. Of the 59 evaluations, only 11 were found which assessed the outcomes of care. The authors note that nine of these 11 studies reported significant improvements in patient outcomes. However, they noted that the size of the improvement varied considerably across the studies.

A 1997 study by Worrall, Chaulk and Freake14 assessed the evidence for CPGs improving patient care outcomes, but only in the realm of primary care conducted by physicians. They reported that just five of the 13 trials identified produced statistically significant results, and that there was insufficient evidence to support the assertion that CPGs improve patient outcomes in primary medical care. However, they specified that a major limitation in their work was that they did not attempt to assess whether the guideline followed in the published trial adhered strictly to the principles of evidence-based medical care and that most of the studies of CPGs published up to that point studied CPGs that were produced before the advent of evidence-based guideline development. Additionally, they reported that only two of the 13 trials reviewed used what they considered to be appropriate methods or enrolled enough subjects to produce sound results. They concluded that there was a clear need for more research that included measures of patient outcomes.

A 1999 study by Thomas et al.15 is a systematic review of 18 studies on the use of CPGs in what the authors described as professions allied to medicine. Studies were included if they reported measurements in process or outcomes of health care, as well as economic outcomes. The authors noted the reporting of study methods was inadequate for all studies and that of the 18 studies, all but one had nurses as the targeted profession. In spite of this, their conclusion was that there was some evidence that guidelines can improve patient care outcomes.

A 2004 study by Bahtsevani, Uden and Willman16 is a systematic review of 10 articles evaluating the outcomes of EB-CPGs. Presumably taking their cue from the Worrall study, these authors specifically made a point of evaluating the clinical strength of the guidelines in each study they reviewed and found the strength of evidence to be weak. They concluded that there was a tendency toward support for the idea that EB-CPGs improve patient outcomes.

A 2009 study by Lugtenberg, Burgers and Westert17 is a systematic review looking at the evidence for the effectiveness of exclusively Dutch, evidence-based clinical guidelines, published between 1990 and 2007, in improving the quality of care. Twenty studies were reviewed, nine of which measured patient outcome data. Of those nine studies, six showed what the authors classified as significant but small improvements in at least some of the outcomes studied, and in the remaining three, no effects on patient healthcare outcomes were observed.

Additionally, in 2013, Ramirez-Morera et al.18 registered their intent to perform a systematic review entitled Effects of evidence-based clinical practice guidelines in health care quality improvements in the PROSPERO prospective register of systematic reviews. This systematic review was scheduled for completion in June 2013. However, no work beyond registering the intent to conduct the review has been reported and it is assumed that this project is not currently active. Despite this, the study protocol entered was extremely thorough, and the authors of this paper acknowledge the use of this protocol in informing the development of this paper. As such, an up-to-date systematic review is required.

Inclusion criteria

Types of participants

This systematic review will include patients of a health care provider, in any setting, provided that that healthcare provider has based their care on EB treatment oriented CPGs. Therefore, participants will be recognized healthcare providers who are in some way responsible for the care of patients (including, but not limited to, doctors, nurses, paramedics, midwives, dental practitioners, optometrists, physiotherapists, psychologists, etc.)

Types of interventions

The review will consider studies in which the primary intervention under investigation is the use of evidence-based treatment-oriented, clinical practice guidelines.

Clinical practice guidelines will be assessed to ensure that they are evidence-based by searching for confirmation that the recommendations of the CPG are directly linked with evidence that has been evaluated and graded in a formalized way for validity (e.g. using the GRADE or FORM evaluation tools).

The use of CPGs in care will be compared with health professionals delivering patient care where treatment decisions have been based on information other than that from an evidence-based, treatment-oriented clinical practice guideline.

Outcomes

Any and all patient outcomes reported by the included studies.

Types of studies

Only experimental or quasi-experimental studies will be considered. Therefore, studies including, but not limited to the following will be sought for inclusion: randomized control trials, clustered controlled trials, interrupted time series and/or controlled before- and after-studies.

Search strategy

The search strategy aims to find both published and unpublished studies. A three-step search strategy will be utilized in this review. An initial limited search of MEDLINE and EMBASE will be undertaken followed by analysis of the text words contained in the title and abstract, and of the index terms used to describe pertinent articles. A second search using all identified keywords and index terms will then be undertaken across all included databases. Third, the reference list of all identified reports and articles will be searched for additional studies. As articles earlier than this will have been captured by earlier systematic reviews, studies published after 2001 will be considered for inclusion in this review, as the most recent review16 had a cut-off date of early 2002. Only studies that are written in English and which examine human patient outcomes will be evaluated. The Database of Abstracts of Reviews of Effects will be searched for related reviews, and PROSPERO will be searched for existing systematic review protocols.

The databases to be searched include:

* Cochrane Central Register of Controlled Trials including the Cochrane Effective Practice and Organization of Care Group Specialized Register

* PUBMED

* EMBASE

* CINAHL

* Google Scholar

* Scopus/Web of Science

* PsychInfo.

Initial key words for PUBMED and EMBASE searches are listed in Appendix I.

Assessment of methodological quality

Quantitative papers selected for retrieval will be assessed by two independent reviewers for methodological validity prior to inclusion in the review using standardized critical appraisal instruments from the Joanna Briggs Institute Meta Analysis of Statistics Assessment and Review Instrument (JBI-MAStARI) (Appendix II).

Any disagreements that arise between the reviewers in relation to the assessment of methodological quality of studies will be resolved through discussion or by a third reviewer.

Data extraction

Data will be extracted from papers included in the review using the standardized data extraction tool from JBI-MAStARI (Appendix III). The data extracted will include specific details about the interventions, populations, study methods and outcomes of significance to the review question and specific objectives.

Data synthesis

Quantitative data will, where possible, be pooled in statistical meta-analysis using JBI-MAStARI. All results will be subject to double data entry. Effect sizes expressed as relative risk for cohort studies, and odds ratio for case control studies (for categorical data), weighted mean differences (for continuous data) and their 95% confidence intervals will be calculated for analysis. A random effects model will be used and heterogeneity will be assessed statistically using the standard chi-square, where statistical pooling is not possible and the findings will be presented in narrative form including tables and figures to aid in data presentation where appropriate.

Acknowledgements

The author is grateful to Australian Catholic University for its support in kind in the development of this protocol.

Appendix I: Search strategies

Initial search string for Pubmed:

Search ((((((((((("Practice Guidelines as Topic"[Mesh] OR "Practice guideline" OR "Practice guidelines")) AND ("2002/01/01"[PDat]: "2015/12/31"[PDat]) AND Humans[Mesh])) AND ((("Guideline Adherence"[Mesh] OR Adherence OR Compliance OR Follow OR Follows)) AND ("2002/01/01"[PDat]: "2015/12/31"[PDat]) AND Humans[Mesh])) AND ((("randomized controlled trial" OR "randomized controlled trials" OR "random allocation" OR "double-blind method" OR "single-blind method" OR "doubleblind method" OR "singleblind method" OR "double blind method" OR "single blind method")) AND ("2002/01/01"[PDat]: "2015/12/31"[PDat]) AND Humans[Mesh])) AND ((("exp research design" OR "exp clinical trials" OR random*[tiab])) AND ("2002/01/01"[PDat]: "2015/12/31"[PDat]) AND Humans[Mesh])) AND ((("Therapeutics"[Mesh] OR Therapeutics OR Therapeutic OR Treatment OR Treatments OR Therapy OR Therapies)) AND ("2002/01/01"[PDat]: "2015/12/31"[PDat]) AND Humans[Mesh])) AND ((("Evidence-Based Medicine"[Mesh] OR "Evidence Based Medicine" OR "Evidence-Based Medicine" OR "Evidence Based Practice" OR "Evidence-Based Practice" OR EBM OR EBP)) AND ("2002/01/01"[PDat]: "2015/12/31"[PDat]) AND Humans[Mesh])) AND ((("Treatment Outcome"[Mesh] OR "Quality of Health Care"[Mesh] OR "Clinical Effectiveness" OR "Clinical Effectivenesses" OR "Clinical Efficacy" OR "Treatment Effectiveness" OR "Treatment Efficacy" OR "Patient outcome" OR "Patient outcomes" OR "Health Care Quality" OR "Quality of Healthcare" OR "Healthcare Quality")) AND ("2002/01/01"[PDat]: "2015/12/31"[PDat]) AND Humans[Mesh])) AND ((("Outcome Assessment (Health Care)"[Mesh] OR "Outcomes Assessment" OR "Outcomes Assessments" OR "Outcome Studies" OR "Outcome Study" OR "Outcomes Research" OR "Outcome Measures" OR "Outcome Measure")) AND ("2002/01/01"[PDat]: "2015/12/31"[PDat]) AND Humans[Mesh])) AND (((Improve* OR Effects OR Effective OR Effectiveness OR Impact OR Impacts)) AND ("2002/01/01"[PDat]: "2015/12/31"[PDat]) AND Humans[Mesh]) Filters: Publication date from 2002/01/01 to 2015/12/31; Humans

Initial search string for EMBASE

'practice guideline' OR 'practice guideline'/exp OR practice AND guideline*:ab,ti OR 'practice guideline'/de AND [english]/lim AND [2002-2015]/py AND [humans]/lim

'protocol compliance' OR 'protocol compliance'/exp OR protocol AND compliance:ab,ti OR 'protocol compliance'/de AND [english]/lim AND [2002-2015]/py AND [humans]/lim

'randomized controlled trial' OR 'randomized controlled trial'/exp OR 'randomized controlled trial'/de