In many healthcare situations, it is common to find several primary studies that have attempted to answer similar questions, yet the differing populations and settings, and contradictory findings, make an overall understanding of the results difficult to apply in clinical practice. Meta-analysis, the statistical technique used in systematic review, provides a solution to this dilemma. A systematic review synthesizes a relevant body of research to "make clear what is known and not known about the potential benefits and harms of alternative drugs, devices, and other healthcare services, and combines the results from many primary studies, usually randomized controlled trials (RCT)" (Eden, Levit, Berg, & Morton, 2010). In this way, the overall effectiveness of a specific healthcare intervention can be determined. The appeal of meta-analysis is that it combines all available research on one topic into a larger study with many participants, increasing the power of the findings. By pooling the results from smaller studies into a larger study, a meta-analysis provides a precise estimate of the outcome of the intervention. A meta-analysis can be used as a guide to answer the question: "Does what we are doing make a difference to X?" even if "X" has been measured across a range of different populations and settings. By statistically combining study results, a common treatment effect of one intervention compared with another (usually the standard of care or a control intervention) and the magnitude of that treatment effect can be calculated.

Meta-analysis is an essential component of a systematic review or a comparative effectiveness review. As such, a meta-analysis uses a similar rigorous methodology as is required of experimental research studies (DeCoster, 2009). Like any other research process, a number of steps are required (see Table 1). These steps include the formation of a research question, searching the literature, appraising the quality of primary studies, extracting data, calculating effect sizes, analyzing data, drawing conclusions, and reporting results (Holly, Salmond, & Saimbert, 2012; Polit & Beck, 2012).

Table 1. Steps in a Meta-Analysis

The process of combining the results of primary studies increases the statistical power of detecting a true relationship between an intervention and a control and provides results that are more objective than those provided by a narrative review. Many primary studies are conducted on small samples and their results do not show statistical significance for the effectiveness of the intervention. For example, in Yang et al. (2012), the total sample size for the meta-analysis was 1044 patients. However, the range in the 12 primary studies was 52-455 patients. By combining study results into a meta-analysis, a larger, more powerful sample was realized and the effect size calculated was more credible.

A meta-analysis can be undertaken to answer clinical practice questions from either a narrow focus or a broad focus (see Table 2). Questions from a narrow focus look at comparing two interventions using a common outcome to determine the overall effect of the new intervention on a specific population, for example, mortality, fall rates, or surgical site infection. The Yang et al. (2012) study is an example of a narrow focus as their outcome of interest was complications following MITHA. However, a review can also answer broader questions than those in individual studies. Questions with a broad focus could look at numerous interventions or treatments for a common condition to identify the best intervention; this method would likely include multiple meta-analyses using subgroups and combining similar interventions together. Broad questions could also explore diverse interventions that use a common outcome measure to answer the question of whether a type of intervention works. While a meta-analysis from a broad perspective can seek to determine an overall class effect of an intervention, this is done at the expense of precision to generalize the results to a broader population (Naylor, 1988). For example, Fransen and McConnell (2008) examined the effects of solid surface exercise interventions designed to reduce pain and improve physical function in patients with osteoarthritis. The exercise interventions varied among the included studies and included aerobic exercise programs, walking programs, muscle strengthening programs, and tai chi classes. Programs were also administered in varying formats including individual attention, group classes, or self-administered at home and over varying periods of time. The authors performed an overall meta-analysis as well as subgroup analyses and concluded that solid surface exercise interventions had at least short-term benefits.

Table 2. Example of a Narrow and Broad Focus in Systematic Reviews

Advantages of a Meta-Analysis

Meta-analyses can increase the external validity or the ability to generalize conclusions. Primary studies that have high internal validity, where the outcome is attributed to the intervention delivered, may lack in external validity. Studies selected for inclusion in a meta-analysis undergo a rigorous appraisal process to ensure that there are minimal threats to the internal validity. Combining primary studies of varying sample sizes and levels of significance increases the variation in the overall sample, allowing the results of the meta-analysis to be generalized to a wider population (Holly et al., 2012).

Disadvantages of a Meta-Analysis

The main criticism about meta-analyses is that authors attempt to combine "apples and oranges" (Polit & Beck, 2012, p. 655). The decision on which study results should and should not be combined is left to the subjective judgment of the authors. The main assumption when combining the results of primary studies is that the studies are homogeneous in terms of populations, interventions, controls, and outcomes. Although there are inherently differences between primary studies that address a common clinical question, those differences should not be so vast that combining their results produces meaningless conclusions. However, the questions asked by a systematic review, and answered through meta-analysis, are often broader in nature than those addressed in individual studies. Combining studies of various fruit may contribute valuable information to answer the question at hand (Borenstein, Hedges, Higgins, & Rothstein, 2009). It is inherent in the nature of a meta-analysis to allow the reviewer to formally investigate these differences through subgroup analysis.

Meta-analyses are criticized as losing the qualitative distinctions between individual studies (Holly et al., 2012). These may be more readily apparent in a narrative review. A meta-analysis, however, does not discount these qualitative distinctions, but instead codes them as moderating variables allowing for their influences to be empirically tested (DeCoster, 2009).

Another criticism of meta-analyses occurs when a reviewer attempts to combine studies of varying effects. If there were wide variances in the effect sizes among primary studies, for example, half the studies favoring the intervention group and half the studies favoring the control group, combing the results in meta-analysis would not be appropriate as the combined effect size would be misleading (Polit & Beck, 2012). Assessing for these variations, called heterogeneity, between primary studies is an important step in conducting a meta-analysis. In this case where heterogeneity in study results is present, it may be more beneficial to explore the difference in study results through subgroup analyses or in a narrative review.

Meta-analyses may contain a biased sample of studies that answer a particular clinical question (Borenstein et al., 2009). When assessing primary studies for inclusion in the meta-analysis, biases, such as selection, performance, detection, and attrition biases, should be carefully evaluated for their effect on the study design. In addition, every attempt should be made to locate all published and unpublished studies that answer the question of interest to limit the effects of publication bias, where only favorable studies are published and unfavorable results get left in the researcher's file drawer. By combining results of both published and unpublished studies and those with significant and nonsignificant results, the true effect size of an intervention can be determined, providing substance to any recommendations made as a result of the meta-analysis findings.

Many of the criticisms of meta-analyses are also true for narrative reviews (Borenstein et al., 2009). It is the systematic approach and transparency in conducting a meta-analysis that helps address the innate challenges in combining results of primary studies and creating meaningful conclusions.

Interpreting the Outcomes of a Meta-Analysis

The results of a meta-analysis are graphically displayed in a Forest Plot (see Figure 1). In a Forest Plot, the effect size and their confidence intervals of primary studies and the summary effect of the combination of these studies are graphically represented as favoring either the intervention or the control. The weight given to the effect size indicates the influence a study has on the combined treatment effect (Holly et al., 2012). Weight is a factor of sample size; studies with larger samples usually have greater precision and therefore are given more weight. A meta-analysis provides for a means to explore differences in effects between studies and to identify heterogeneity among study results (The Joanna Briggs Institute, 2011). There are several questions to ask to interpret the Forest Plot (see Table 3). These questions are used to determine whether recommendations based on the results of a meta-analysis are best practice.

Figure 1. Patient self-reported functional improvement at 6 months. From "Is Surgical Intervention More Effective Than Non-surgical Treatment for Carpal Tunnel Syndrome? A Systematic Review," by Q. Shi and J. C. McDermid, 2011, Journal of Orthopaedic Surgery and Research,

Table 3. Questions for Interpreting a Forest Plot

The Forest Plot in Figure 1 presents the results of a meta-analysis on the effectiveness of surgical versus nonsurgical treatment for carpal tunnel syndrome (CTS; Shi & McDermid, 2011). Although surgical intervention is considered as the definitive treatment for CTS, it is not always considered first. Conservative intervention, such as drugs, physical therapy, or wrist splints, may provide sufficient relief and may be a patient preference due to concerns about the discomfort, inconvenience, or safety of surgery (Shi & McDermid, 2011).

There are seven studies in this systematic review; five are randomized controlled trials and two are controlled (not randomized) trials. The meta-analysis in Figure 1 is the combined results of four of these seven studies on patients' self-report of functional improvement 6 months following conservative or surgical treatment for CTS. This is an analysis of a subgroup of studies that included data related to the outcome of interest: patients' self-report on functional improvement. Included in this analysis were 416 total subjects: 199 in the surgical group and 217 in the nonsurgical group. The summary effect can be determined by looking at the placement of the diamond-shaped graphic. The vertical dark line at zero (0) on this graph is the null hypothesis. In other words, if the review results (represented by the diamond) cross this vertical line, the results are not significant. However, in this study, the diamond is fully on the left, indicating that the results of this meta-analysis are significant. It was concluded that surgical intervention had superior benefit over conservative treatment based on patient self-reports 6 months after treatment.

Borenstein M., Hedges L. V., Higgins J. P., Rothstein H. R. (2009). Introduction to meta-analysis. West Sussex, UK: Wiley. [Context Link]

DeCoster J. (2009). Meta-analysis notes. Retrieved June 2, 2012, from http://www.stat-help.com/notes.html[Context Link]

Eden J., Levit L., Berg A., Morton S. (2010). Finding what works in health care: Standards for Systematic Review. Washington, DC: National Academies Press, Institute of Medicine. [Context Link]

Fransen M., McConnell S. (2008). Exercise for osteoarthritis of the knee. Cochrane Database of Systematic Reviews, 4, CD004376. doi: 10.1002/14651858.CD004376.pub2 [Context Link]

Holly C., Salmond S. W., Saimbert M. K. (2012). Comprehensive systematic review for advanced nursing practice. New York: Springer Publishing Company. [Context Link]

Naylor C. D. (1988). Two cheers for meta-analysis: Problems and opportunities in aggregating results of clinical trials. Canadian Medical Association Journal, 1387, 891-895. [Context Link]

Polit D. F., Beck C. T. (2012). Nursing research: Generating and assessing evidence for nursing practice. Philadelphia, PA: Lippincott Williams & Wilkins. [Context Link]

Shi Q., McDermid J. C. (2011). Is surgical intervention more effective than non-surgical treatment for carpal tunnel syndrome? A systematic review. Journal of Orthopaedic Surgery and Research, 6, 17. doi:10.1186/1749-799X-6-17 [Context Link]

The Joanna Briggs Institute. (2011). Joanna Briggs Institute reviewer's manual: 2011 edition. Adelaide, Australia: Author. [Context Link]

Yang B., Li H., He X., Wang G., Xu S. (2012). Minimally invasive surgical approaches and traditional total hip arthroplasty: A meta-analysis of radiological and complications outcomes. PLoS ONE, 7(5), e37947. doi:10.1371/journal.pone.0037947 [Context Link]

For 37 additional continuing nursing education activities on research topics, go to http://nursingcenter.com/ce.