1. Keith, Jessica L. DNP, RN, PCCN
  2. Doucette, Jeffrey N. DNP, RN, CENP
  3. Zimbro, Kathy PhD, RN
  4. Woolwine, David MEd, SPHR

Article Content

In 2010, the U.S. Congress passed the Affordable Care Act (ACA) in an effort to control the rising cost of healthcare and reduce the number of individuals who are uninsured. The ACA is a several-step plan that will be phased in over a 4-year period. The primary goal of the ACA is to ensure that all recipients of healthcare receive quality care at an affordable price based on evidence and outcomes.

Figure. No caption a... - Click to enlarge in new windowFigure. No caption available.

The Centers for Medicare and Medicaid Services (CMS) is a government entity that sets guidelines for hospitals to follow to receive maximum payment reimbursement. Over the past decade, the CMS has restructured its reimbursement system based on various initiatives known as pay-for-performance or Value-Based Purchasing, which are parallel to the ACA's goals.1 The CMS measures clinical outcomes and the overall patient experience to ensure that quality care is delivered to its beneficiaries.


In order to quantify the patient's experience, the CMS created a standardized survey entitled the Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS). This survey captures patients' impressions of their hospital care following discharge from inpatient settings by asking a core set of 27 questions. These results, listed as summarized composites, are publically reported on the Hospital Compare website. This site allows consumers to view how participating hospitals rank on safety, service, and quality. The results are described on the site as top-box, middle-box, and bottom-box scores. The top-box score represents the 75th percentile for the most positive response to HCAHPS survey questions. In order to maximize reimbursement, the CMS requires hospitals to be within the top-box range for HCAHPS survey questions.2,3


A 210-bed, level III community-based acute care hospital with a greater than 50% Medicare/Medicaid population implemented the IMPACT program to improve its patient experience and HCAHPS scores after underperforming on HCAHPS surveys during the previous 3 years. This study evaluates the program's impact on the HCAHPS scores at the study site.



The IMPACT program incorporates situational leadership and is designed to reduce variation in care and improve the patient experience by providing excellent customer service to every patient, every time. The program focuses on leader involvement in assessing, coaching, and holding staff accountable for providing consistency in customer service delivery using the nursing and ancillary service bundles.


An 8-hour onsite seminar was developed for hospital leaders based on the situational leadership assessment model created by the Center for Leadership Studies. Situational leadership engages leaders in evaluating their staff members based on how well they perform a task. For the IMPACT program, the tasks were divided as nursing and ancillary service bundles. Nursing bundles included hourly rounding, bedside shift report, use of whiteboards in the patient rooms, communication with the patient, timeliness of addressing the patient's needs, explaining adverse reactions of medications, and overall patient experience. The ancillary service bundles included room cleanliness and overall patient experience. All of these bundles had previously been incorporated at the study hospital; however, consistency in delivering each of these components varied by healthcare professional.


IMPACT tenets teach leaders to adjust leadership style based on performance of specific nursing care components. Staff members were engaged in rapid cycle change to improve patient care and patients' perceptions of customer service. The training seminar occurred in February 2013 with mandatory participation by hospital leadership. The seminar was well received by the leaders; they felt compelled to assist in making the program tenets come to fruition.


At the conclusion of the 8-hour IMPACT training seminar, the leaders were instructed to educate their staff members about the "gold standard" for nursing and ancillary service bundles to set a standardized expectation for everyone to follow. The "gold standard" was defined by the organization and deemed to be performance excellence every time with every patient interaction. The education was to be completed by March 15, 2013. Before educating the staff, leaders were tasked with evaluating each member of their team according to his or her ability to consistently perform the bundled tasks to the "gold standard"; this evaluation would allow the leader to tailor the education based on the individual's readiness level. The initial evaluations of staff readiness were completed by the leaders on February 28, 2013. Staff members had mixed emotions about the program tenets because many believed they were already performing at the "gold standard" and felt this was just another initiative created by the hospital leadership teams.


Thirty days after leaders educated staff members on the "gold standard" expectations, they reevaluated staff performance using the situational leadership assessment model. Any employee who wasn't consistently meeting or performing at the "gold standard" was placed on a corrective action plan for 60 days. Many staff members placed on action plans felt they were being targeted whereas others viewed it as a growth opportunity. Several leaders were uncomfortable with the action plan requirement because they feared it would cause additional turnover when resources were already slim. Although the leaders understood the need to hold staff accountable, many had empathy toward the staff and needed to remove the "face" from the action plan. It wasn't about how long an employee had been there but how willing and able he or she was to meet the "gold standard" expectation. It took strong leadership to look at the variations and have crucial conversations with the underperforming employees.


Throughout the 60-day action plan, leaders provided individualized coaching to staff members based on the specific bundle component they weren't meeting. At the end of the 60 days, mid June 2013, leaders reevaluated the staff on action plans to establish if sustained improvement had been achieved. If not, the action plan was revised and coaching continued for an additional 60 days. On August 15, 2013, the leaders did a final evaluation of the staff on action plans; if the individual was inconsistent in meeting the "gold standard" and improvements hadn't been made, they were terminated. At the study hospital, eight employees were terminated for not meeting and sustaining consistency in the "gold standard" for the bundle components and eight employees voluntarily resigned due to dissatisfaction with policies and work environment. (See Table 1.)

Table 1: IMPACT prog... - Click to enlarge in new windowTable 1: IMPACT program checklist and action items

Study design

The study was a retrospective descriptive design that assessed whether there was a difference in specific HCAHPS survey questions following the implementation of the IMPACT program. It aimed to: (1) assess the impact of situational leadership training on scores specific to the HCAHPS survey questions related to the patient experience and (2) assess whether there was a difference in the scores of specific HCAHPS survey questions after leader participation in a situational leadership training session. The study set out to address three evidence-based research questions: (1) Is there a difference in the top-box scores for overall hospital rating following implementation of the IMPACT program? (2) Is there a difference in the top-box scores for the service bundle (room cleanliness) following implementation of the IMPACT program? (3) Is there a difference in the top-box scores for the nursing bundle (pain well controlled, help going to the bathroom as soon as wanted, got help as soon as wanted, nurses listened carefully, nurses explained things understandably, and nurses treated you with courtesy/respect) following implementation of the IMPACT program?


This research study described IMPACT as the independent variable. The dependent variables in this study were the pre- and posttraining session HCAHPS survey scores for specific questions related to nursing and ancillary bundles. Specifically, 8 of the 27 HCAHPS survey questions were examined for differences in patients' perceptions of nursing care and overall hospital satisfaction. (See Table 2.)

Table 2: Research qu... - Click to enlarge in new windowTable 2: Research questions and corollary HCAHPS survey questions

After surveys were collected, results were reported by discharge unit. Participants included patients who returned the HCAHPS surveys during the study timeframes from pre- (July 1, 2012 to January 31, 2013) to posttraining (March 1, 2013 to September 30, 2013). All research questions were analyzed and the results were then categorized as "care ratings across units" and "care within units."


Care ratings across units

In order to examine significant differences from pretraining to posttraining across all hospital units' ratings, chi-square tests of independence were conducted with all patient surveys included. In these total patient analyses, alpha was set at 0.05 for each question addressed. Analyses revealed that ratings for hospital satisfaction ([chi]2(1) = 2.02, p = 0.156, [PHI] = .031), nurse courtesy ([chi]2(1) = 0.00, p = 0.988, [PHI] = 0.000), nurse explanation ([chi]2(1) = 1.38, p = 0.239, [PHI] = 0.025), following bathroom protocol ([chi]2(1) = 0.84, p = 0.360, [PHI] = -0.025), pain control ([chi]2(1) = 0.48, p = 0.488, [PHI] = 0.018), and room cleanliness ([chi]2(1) = 3.69, p = 0.055, [PHI] = 0.041) weren't significantly different before and after training. However, ratings for nurse listening behavior ([chi]2(1) = 4.28, p = 0.039, [PHI] =0.044) and patients' receipt of help ([chi]2(1) = 4.54, p = 0.033, [PHI] = 0.049) after the training were significantly improved relative to pretraining ratings, when examining all patient ratings across units.


Care ratings within units

In order to examine significant differences within each of the units' ratings, chi-square tests of independence were conducted within each unit. (See Table 3.) In consideration of potential Type 1 error inflation, given that multiple inferential tests were conducted for each unit, alpha was set at 0.007 for each HCAHPS question that was evaluated. The alpha correction was calculated by dividing alpha of 0.05 by 7 (the number of within unit comparisons conducted for each outcome). Given the exploratory nature of this research, clinically significant differences are identified within specific units regardless of statistical significance.

Table 3: Changes in ... - Click to enlarge in new windowTable 3: Changes in top-box score by unit

None of the units demonstrated statistically significant changes in hospital satisfaction ratings. Units didn't demonstrate statistically significant differences in nurse courtesy ratings. However, two units' courtesy ratings, unit B and unit E, decreased 6.1 and 11.9 points, respectively; whereas unit D's ratings increased 5.4 points. Units didn't show statistically significant differences in nurse listening ratings. Although not significant after alpha corrections, it's notable that unit D's listening ratings increased 9.2 points, and unit C and unit E's ratings increased approximately 5 points. Of all units, only unit C demonstrated statistically significant differences in nurse explanation ratings as shown by its 11.6-point increase.


Clinically notable, unit E's explanation rating increased 15.8 points. Although changes in bathroom protocol ratings following the training weren't statistically significant, unit F's ratings decreased 13.4 points and unit E's ratings decreased by almost 7. Unit A and unit D's bathroom ratings increased approximately 8 points each. However, unit F's ratings decreased 13.4 points, and unit E's ratings decreased 6.6 points. Pain control ratings didn't change significantly in any particular unit. Although not significant, unit E's pain control ratings increased 28.2 points; similarly, unit F and unit C's pain control ratings increased approximately 9 points. Room cleanliness ratings within units didn't demonstrate statistically significant changes. However, unit F's room cleanliness ratings increased 11.9 points; similarly, unit C and unit A's room cleanliness ratings increased approximately 6 points. None of the units' ratings for receiving help demonstrated statistically significant changes. Notably, unit E's help ratings increased 16.2 points, and unit C and unit A also increased 8.1 and 9.8 points, respectively.



This study had several limitations, which could be examined in future research. First, the rate of returned surveys was less than 40% for all units combined; approximately 15% were undelivered due to incorrect or missing addresses and the rest were left unanswered. The total number of surveys returned during the pretraining was 34.6% of the actual surveys sent. During the posttraining period, 36.5% of the surveys sent were actually returned.


A second limitation noted was the frequency of missing data. Patients who chose to answer the survey weren't required to answer all questions; furthermore, if an item was left unanswered, it was difficult to ascertain whether it was applicable or comprehended.


Third, the analysis for this study only looked at the relationship of top-box score differences from pre- to posttraining. It didn't take into account changes between ratings, such as "sometimes" ratings changing to "usually" ratings in the postperiod. Examining at this level would show differences although they may not have fallen in the desired top-box range.


Fourth, the length of time to evaluate the posttraining period may not have been adequate enough. Although the HCAHPS scores were examined during the 6 months following the seminar, the staff members who were on the action plans were still being coached during that time. A longer period of time may more accurately reflect the changes in culture that needed to be established to truly impact scores.


Lastly, individual unit sample sizes were small and returned surveys were even smaller. Future research should allow for data analysis on a per unit basis and further investigation on specific leader strategies that impacted the customer service scores. Clinical significance could be expounded in future research to assess the readiness levels of the team and how effective the leader was in adapting his or her leadership style to match staff members' ability to provide an exceptional patient experience.


Accountability is key

Leaders must evaluate their customer service processes, assess gaps, and create solutions to improve the patient experience. The leadership team acknowledged that significant improvements wouldn't be immediately available because culture change takes time. The posttraining period, although not statistically significant, did reveal that changes in the right direction were occurring. The team would like to reexamine the data after a full year to evaluate whether the culture changes were sustained and the "gold standard" was being performed. Because the IMPACT program is in the infancy stages of development, it hasn't proven to be evidence-based, but tenets may offer opportunities to improve the patient experience and achieve maximum CMS reimbursement for services delivered.


At the conclusion of this study, the site has continued to see improvements in the HCAHPS scores. A future study might reveal a more statistically significant difference in the scores as variation is reduced and accountability is sustained. The leadership team continues to place emphasis on leaders holding staff accountable to perform at the "gold standard". Leaders remain diligent with ongoing evaluations of staff members against their readiness level using the situational leadership model. Leaders are working directly with their staff members as coaches and mentors to ensure they meet the expectations. The key to a successful work environment that strives for excellence is reduced variation and strong accountability.




1. Raso R. Value-based purchasing: what's the score? Reward or penalty, step up to the plate. Nurs Manage. 2013;44(5):28-34. [Context Link]


2. MacDavitt K, Cieplinski JA, Walker V. Implementing small tests of change to improve patient satisfaction. J Nurs Adm. 2011;41(1):5-9. [Context Link]


3. Long L. Impressing patients while improving HCAHPS. Nurs Manage. 2012;43(12):32-37. [Context Link]