Since its inception in 2008, the American College of Surgeons Trauma Quality Improvement Program (ACS TQIP) has been improving the quality of care for trauma patients ("Level I & II TQIP," n.d.; Nathens, Cryer, & Fildes, 2012). The TQIP collects data from Level I and Level II trauma centers and provides feedback to each TQIP center via risk-adjusted benchmarking. These measures provide a scientific basis in which participating centers can enact changes to clinical practice ("Level I & II TQIP," n.d.).
The TQIP reports mortality and complications for all TQIP patients. As of the Fall 2017 TQIP benchmark report, the TQIP reported on the following 10 cohorts of patients: (1) blunt multisystem injuries, (2) elderly patients, (3) elderly patients with blunt multisystem injuries, (4) elderly patients with isolated hip fractures (IHFs), (5) fractures, (6) hemorrhagic shock, (7) penetrating injuries, (8) severe traumatic brain injuries, (9) shock, and (10) splenic injuries (Table 1). For each of these cohorts, statistical models are used to create risk-adjusted estimates for outcomes and complications.
The purpose of this article is to provide a guide to trauma programs that demonstrates how using the TQIP hospital-specific data and benchmarking to national data can improve outcomes. We discuss data provided by the TQIP and how to identify trends and outliers in the data, create individualized solutions and corrective action plans, and reevaluate the data to assess the impact of the corrective action plan. We review this process using examples from our institution that highlight 4 categories by which data and reports should be evaluated: processes of care (i.e., time to operating room [OR]), quality of care (i.e., protocol adherence), data coding (are issues coded according to definition), and data mapping (linkages within the system). This project was reviewed by the hospital's institutional review board and was deemed "not human subject research."
STEP 1: EVALUATE TRENDS AND OUTLIERS IN THE DATA
All TQIP statistical model estimates are presented as odds ratio(s), which allows the individual TQIP-enrolled hospital to compare their data with all TQIP participating hospitals, while controlling for relevant patient and injury characteristics. The odds ratio is a measure of association between an exposure and an outcome (Szumilas, 2010). When an odds ratio is above 1, the odds of an outcome are greater than average, whereas when an odds ratio is below 1, the odds of an outcome are less than average. The box plots provided in the TQIP report are excellent visual depictions of the odds ratio data; the box plots allow the user to see, at a glance, if his or her institution is above or below the median (Figure 1). In addition, if the confidence intervals are both above or both below the median line, the hospital is considered a high or low outlier, respectively. The three circled box plots in Figure 2 demonstrate high outliers where the odds ratio and confidence intervals are above the median.
In addition to identifying outliers at one point in time, the TQIP data can be used to track trends over time. Viewing the line graph that covers nine reporting periods (4 years of data) enables the user to track whether his or her performance has been improving or declining for specific outcomes and complications (Figure 3). We added a red arrow to Figure 3 to demonstrate a worsening performance where the odds ratios are increasing and a green arrow to show improvement or decreasing odds ratios.
Once an outlier or trend has been identified, one needs to consider which category is primarily responsible for the poor or worsening performance. Is it a process of care issue, a quality issue, incorrect data coding, or a data mapping issue? Use the drill down feature in Trauma Quality Program (TQP) Explorer (as outlined in Table 2) to obtain individual patient data, such as demographics, clinical characteristics, and outcomes. Identifying the patients in the problem area, potentially performing retrospective chart reviews to obtain additional details, and analyzing the individual patient-level data are the essential next steps that can provide further insights into what is causing the problem and where improvements can be made.
STEP 2: CREATE INDIVIDUALIZED SOLUTIONS
Sharing the results with the institution's trauma leadership will bring attention to the area needing improvement and start the process of developing hospital-specific solutions. First, share the results with the trauma program staff, multidisciplinary committees, and hospital administration. Then, form a committee of interested stakeholders to create a corrective action plan. Potential action items of the committee may include creating or revising clinical care guidelines, reviewing definitions of specific complications with the team, and instituting prophylaxis or other measures to reduce those specific complications, providing education for interrater reliability purposes, and double-checking software data mapping, if necessary.
STEP 3: REEVALUATE THE DATA
The final step in using the TQIP data is to reevaluate the problem area. After the corrective action plan has been implemented, examine the latest TQIP report to see whether the complication rate originally identified as a problem has improved. This will help determine whether the action items that were applied have made an impact or whether it is time to refocus the approach; this is an example of loop closure. TQIP reports are published twice a year; it may require several cycles to see changes to the TQIP numbers after remedial actions have been implemented.
EXAMPLES
At our institution, TQIP data have been used regularly to track outcomes and improve quality of care. We present four scenarios of how we have used the TQIP data for the past 8 years to effect change in our care of patients and improve outcomes. The first scenario focuses on a process of care issue related to unplanned returns to the intensive care unit (ICU) in IHF patients, the second pertains to quality of care in patients with catheter-associated urinary tract infections (CAUTIs), the third describes a deep vein thrombosis (DVT) data coding issue, and the fourth was a data mapping problem involving splenectomy procedures.
Each example showcases surveillance activities, interventions, and loop closure, which are important aspects of Performance Improvement and Patient Safety (PIPS) programs. Effective PIPS programs identify opportunities for improvement that lead to specific interventions to ensure future patients will not experience the same complication. The plan of action in each example is the specific intervention that ensures these events are less frequent. The loop closure activity is utilizing the TQIP report to monitor progress toward reducing the targeted complications.
Process of Care: Isolated Hip Fractures
The TQIP data demonstrated an increase in major complications among IHF patients 65 years or older at our trauma center in comparison with other similar facilities (Figure 3) over two cycles despite process changes. This indicated a trend that required major investigative work. Our objective was to investigate the IHF patients with major complications in order to identify trends and create specific corrective action plans.
By using the TQP Explorer, we examined the types of complications patients experienced; the majority were an unplanned return to the ICU (73%). Consequently, this was our focal group in which to identify trends and create an improvement plan. We exported this specific patient list from the TQP Explorer to an Excel spreadsheet and performed a detailed retrospective case analysis that examined patient management, time to surgery, reason for delay of surgery, whether the patient was initially admitted to the floor and had an unplanned ICU stay or whether the patient was discharged from the ICU but had an unplanned return, and the reason for the unplanned ICU stay. Then, we compared demographics, clinical characteristics, and outcomes between IHF patients with an unplanned return to the ICU and those without major complications. We found that IHF patients with an unplanned return to the ICU experienced a longer time from arrival at the hospital until operative fixation of their hip fracture, thus reinforcing the importance of timely (<48 hr) surgical intervention for geriatric hip fractures ("ACS TQIP Best Practice Guidelines in the Management of Orthopaedic Trauma," 2015). This finding was in agreement with an article published in JAMA that also discussed the importance of time to operative intervention. Their results concluded that the risk of complications increased when the time to OR exceeded 24 hr, irrespective of the type of complication (Pincus et al., 2017).
Once we identified the unplanned return to the ICU as an outlier and noted the time to OR as the relevant independent variable, we had a multipronged attack to address the problem. We incorporated new geriatric clinical care guidelines, which included emergency department (ED) time of arrival to wheels in the OR, mandatory internal medicine/critical care consultation to fine-tune active and chronic comorbidities, and upfront pain management to reduce narcotic delirium/encephalopathy. We began evaluating every unplanned return to ICU patients and discussed the specific care-related issues in Trauma Morbidity & Mortality and Multidisciplinary Quality Improvement meetings. We also convened a working group involving all services in the care of geriatric patients with hip fracture to get agreement on care protocols and guidelines. Our preliminary data show improved ED to OR times and a trend toward decreased unplanned return to the ICU.
Quality of Care: Catheter-Associated Urinary Tract Infections
In this example, we did not use the TQIP data to initially find the problem. Nursing leaders at our institution recognized a high number of CAUTIs when reviewing fiscal year 2014 hospital quality CAUTI data. As a result, the HOUDINI (Hematuria, Obstruction, Urological, gynecological, or perineal surgery patients, Decubitis ulcers-open sacral or perineal wound in an incontinent patient, I&O, strict for critical patients, oN comfort or hospice care, and Immobility) protocol (Adams, Bucior, Day, & Rimmer, 2012), an evidence-based, nurse-driven urinary catheter removal algorithm was approved for use at our institution in 2014. With the HOUDINI protocol, instead of waiting for orders to remove the catheter, the nurse performs daily morning assessments for continued catheter use and removes the catheter if the criteria for continued use are not met. At our institution, implementation of the HOUDINI protocol began in 2015 and all nurses had to complete online learning modules by June 30, 2015. We then used the TQIP data to assess and validate the impact of implementing a process, in this case the HOUDINI protocol, on quality outcomes. We also compared the characteristics of patients with CAUTI with patients without CAUTI.
We used the Spring 2017 TQIP UTI/CAUTI complication data to retrospectively examine 1 year of data after the required nursing education on the HOUDINI protocol was complete (July 2015-June 2016). The number of CAUTIs significantly decreased in the first 6 months after implementing the HOUDINI protocol to the subsequent 6 months (27 CAUTIs to 5 CAUTIs, p< .001). When examining the characteristics and outcomes among CAUTI patients, we found that patients with CAUTI were older, more often female, had a higher injury severity score, and were more likely to use alcohol and steroids. Patients with CAUTI also had more complications including ventilator-associated pneumonia (VAP), increased ventilator days, more days in the ICU, and longer hospital lengths of stay. When examining the CAUTI organisms, it came to our attention that patients with a CAUTI and VAP often had the same organism of infection.
Our TQIP data suggested that implementing the HOUDINI protocol led to fewer CAUTIs at our Level I trauma center. After completing the assessment of these data, we still created a multidimensional plan of action in order to reinforce our initial work and begin to address new findings. First, we provided education on the prevention of CAUTIs and the HOUDINI protocol through an online learning portal to reinforce the importance of the HOUDINI protocol on our CAUTI success. Second, we communicated the risk factors that place patients at our institution at risk for CAUTI to the staff. Third, although we do not know which came first, the CAUTI or VAP, we communicated the finding that the CAUTI and VAP organisms were often the same. This finding needs to be further explored, but we advocated for good handwashing of not only the provider but also the patient in case the organisms are spreading through contact. Finally, a committee at our institution conducts a debriefing on every CAUTI event and we use the electronic health record to monitor indwelling catheters by pulling a daily report of every patient with a catheter for longer than 48 hr so that the patient can be evaluated more efficiently for continued use of the Foley catheter.
Data Coding: Deep Vein Thrombosis
Our institution's first TQIP report showed a high percentage of DVTs. We questioned whether we had a high percentage of DVTs because a baseline screening for DVTs is performed at our institution (process of care) or because the DVT prophylaxis protocol was not being followed (quality of care). In reality, the high number of DVTs was a data coding issue where the specific definition provided by the TQIP was not being followed.
The way we determined that the specific definition provided by the TQIP was not being followed was by first identifying all patients with DVT from the TQP Explorer. These patients were exported to create a subpopulation of DVT patients in our database. Once this was accomplished, a multivariable report was created for the DVT population that included further patient identifiers. After review of all patients with identified clot, all charts were rereviewed to confirm that these patients with clot were treated with therapeutic heparin. After review of multiple cases, it was clear that the National Trauma Data Bank (NTDB) definition was not being followed; patients were being coded as having a DVT complication when they were not being treated with therapeutic heparin. Education was then provided to all data registrars for interrater reliability purposes, and the team was tasked with going back into the database to remove the inappropriately assigned complication. Once the data were coded on the basis of the specific TQIP definition, the complication rate in future biannual reports aligned with the national averages.
To ensure the correct TQIP definition of DVT is utilized, we continue to monitor our work. We check that our DVT rate aligns with national averages and that the rate does not jump up or down unexpectedly from one TQIP report to the next. We also review the TQIP DVT definition with the addition of new staff and/or turnover, and the data registrars now meet monthly to review interrater reliability and review state and NTDB definitions.
Data Mapping: Missing Splenectomies
The fourth example of how we have used the TQIP data to evaluate our trauma program involves the important role of data mapping to our data quality. Review of the Fall 2017 TQIP report identified that for 1 year of submitted TQIP data, the report had not registered any splenectomies at our facility. Knowing this to be an error, we set out to explore why this had occurred.
The first step was to utilize the TQP Explorer to identify the splenic injury cohort and export that list from the TQP Explorer. Second, we imported all pertinent trauma numbers into a specific TQIP spleen injury population in our own Trauma One database. Third, we were able to pull multiple fields into a report to identify trends in the patients who had spleen injury and splenectomy in our Trauma One database system. After running reports from our own Trauma One database, it was clear that splenectomies had been performed at our facility within the TQIP reporting time frame.
During our review, we identified that all splenectomy patients had their operations in our trauma resuscitative OR suite, otherwise known locally as our "T-10 room." In our T-10 room, patients who are physiologically unstable (hypotensive/tachycardic) or who meet mechanistic criteria to have a high likelihood of uncontrolled hemorrhage are taken directly to the OR and bypass the ED. There, diagnostic, resuscitative, and therapeutic hemorrhage control options can be applied by a whole team of individuals. The next step involved reaching out to our software vendor to confirm that the T-10 location is mapped to an OR location, as only splenectomies that occur in an OR are used by the TQIP in their reporting. When this was completed, we discovered that our T-10 location code was being mapped to "6," which translates to OTHER and was therefore not being pulled into the TQIP report.
The TQIP cohort criteria specific to splenic injury require that only splenectomies that have occurred in a procedure location of OR are included in the data pull. If the procedure location is not OR or not mapped to OR behind the scenes, then those cases would be excluded. This is exactly what occurred; it was a problem identified within our own database mapping system and once identified we worked with our vendor to correct the problem. Once corrected, resubmitting data to the TQIP corrects the data in subsequent benchmark report.
Simple data mapping can cause problems that lead to glaring misrepresentations of our cohorts. When adverse trends or high outliers are discovered, investigation by our trauma program manager should include an evaluation of appropriate data mapping and entry as the potential confounding component to our trauma quality.
CHALLENGES
There is a plethora of published articles on the virtues of a national registry to help guide improving quality (Hemmila et al., 2010; Nathens et al., 2012; Shafi et al., 2009). The TQIP is one of many ACS quality programs recognized as a force for quality improvement at the local and national levels. The TQIP's data quality is maintained rigorously by the ACS, but there are challenges in using registry data (Arabian et al., 2015). For example, some metrics are added and some are dropped for a variety of reasons including the scope of the problem, the overall number of affected patients, or when more important parameters become apparent. In addition, conducting a detailed analysis of patients is tedious and labor-intensive and requires adequate time and resources. Obtaining these resources can be a challenging case for trauma program managers to make to their hospital administrations in this cost-conscious health care climate. As technology becomes more sophisticated, such as the TQP Explorer, more training is required to familiarize staff with the added features, which can take time away from the limited resources. Finally, before undertaking time- and resource-consuming projects in hopes of positively affecting their outcomes, program managers must remember to constantly review data entry for errors that might skew their data and interpretation of the results.
CONCLUSIONS
The ACS TQIP was created to provide trauma centers the ability to benchmark their quality of care and outcomes with national averages. One of the ultimate benefits for participating institutions is the ability to assess their performance over time and to see the effects of their initiatives to the quality "bottom line." We believe that if we analyze our TQIP data with a focus on appropriate processes, quality metrics, and accurate coding and data mapping, we will find the TQIP data to be extremely useful for improving patient care. Applying the appropriate resources and expertise to our data can positively improve outcomes and therefore reduce costs.
KEY POINTS
* The ACS TQIP provides trauma centers with the ability to benchmark the quality of their care by providing real-time and retrospective data that can be used to evaluate patient risk profiles.
* It is important to investigate our institution's data and drill down for trends that can be improved through evidence-based practice change.
* Evaluating the impacts of corrective action plans is vital to the sustainment of clinical practice change.
REFERENCES