BACKGROUND AND SIGNIFICANCE
In the hospital setting, maintaining balance between patient demand and caregiver availability is a prerequisite to high-quality care. The current process of work allocation in hospital-based patient care units entails the creation of nurse-patient assignments, typically performed by a charge nurse. In a primary care nursing model, the nurse-patient assignment specifies the set of patients for whom an on-duty RN will assume primary responsibility for nursing care delivery during a work shift. Numerous US states have enacted legislation that require hospitals to maintain staffing plans, specific minimum nurse-to-patient ratios, or other forms of public reporting, as resource insufficiency is associated with missed nursing care,1,2 decreased quality of work performance,3 staff burnout,4 and reduced ability to detect and correct system failures,5 bone fractures,6 and patient mortality.7
Sustaining high-quality care across variable demand8,9 requires dynamic awareness of the balance between patient needs and staff capacity. Information sources used to maintain awareness of workplace conditions include the experience and clinical judgment of registered nurses (RNs) and quantified estimates of work intensity produced by electronic health records (EHRs) or stand-alone patient acuity or classification tools. Recognized challenges to workload estimates include inadequate definitions and descriptions of nursing work and a need for reliable and valid methods for assessing work demands in dynamically changing situations.10
Workload estimation tools embedded in the EHR provide dynamic estimates of work intensity, since these tools assimilate new data entered into the EHR by clinicians throughout a work shift. Baseline logic used to estimate work intensity may be provided by the EHR vendor. Postimplementation evaluation is needed to ensure that the EHR work intensity tool accurately reflects RN workload and supports effective resource allocation, but few evaluations of dynamically generated EHR-generated work intensity scores are published in current literature.
OBJECTIVES
The objective of this study is to describe relationships between EHR-generated work intensity scores and self-reported RN ratings of appropriateness of patient care assignment across a variety of hospital patient care units.
METHODS
This study was conducted on six patient care units within a large academic medical center concurrent with the new implementation of an EHR-based work intensity estimation functionality within an existing EHR system. At the time of evaluation, the work intensity tool was not visible to frontline staff, to avoid the potential for EHR-estimated work intensity scores to influence self-reported RN ratings of appropriateness of patient assignment. Work intensity scores and RN ratings of the appropriateness of patient assignment were collected on six patient care units with differing patient populations. The units included an adult medical intensive care, pediatric intensive care, adult medical-surgical, pediatric medical-surgical, labor and delivery, and mother-baby unit. All units utilize a primary nursing care model, in which a single RN assumes primary responsibility for direct delivery of nursing services for one or more patients. Four of six patient care units exclusively utilize 12-hour work shifts, one uses 8-hour shifts, and one employs a mix of 8- and 12-hour work shifts. The typical ratio of patients to nurses is 1:1 to 2:1 in the intensive care units and labor and delivery. On the mother-baby unit and medical-surgical units, the typical ratio of patients to nurses is approximately 4:1 on day shift and 4:1 to 5:1 on night shift.
Data Collection of Work Intensity Scores
Patient-level work intensity scores are automatically generated by the hospital's EHR (EPIC, Madison, WI). Although the EHR allows for customization of work intensity logic, the workload intensity tool was implemented generically using recommended workload estimation rules provided by the EHR vendor. Work intensity scores for each patient are based on points accrued across a large number of workload estimation rules embedded in the EHR. Types of nursing care represented in workload estimation rules include assessments, admissions and discharges, medications, risks, activities of daily living, wounds, lines, drains, and airways. Contextual aspects of care such as care task frequency are derived from provider orders and nursing documentation. Work intensity scores do not possess a defined unit of measure or predefined maximum value. An RN-level score is generated by summing the work intensity scores of the patients assigned to an individual nurse.
A single EHR-generated work intensity score was exported for each patient at the beginning of each work shift. An RN-level work intensity score was generated by summing the scores of patients assigned to an RN for the work shift. At the time of this study, summation of patient scores into an RN-level work intensity score was performed manually outside the EHR because facility RNs did not consistently utilize the EHR treatment team functionality, which creates this sum automatically. Light-duty RNs performed data entry after received training from an experienced nurse informaticist. Process steps included determining which patients were assigned to an RN using the patient care units' paper-based daily assignment sheet, selecting associated patient-level work intensity scores, and summing patient-level work intensity scores into an RN-level score for each nurse and work shift. Data entry quality assurance was achieved through periodic spot checks of work intensity score accuracy by the informaticist who provided training in data entry procedures.
Data Collection of Registered Nurse Appropriateness Ratings
A single self-reported rating of the appropriateness of patient assignment was verbally collected from on-duty RNs each work shift using a modified question from the Practice Environment Scale of the Nursing Work Index.11 Response options to the question "During the first 4 hours of my work shift, my patient care assignment was appropriate, considering both the number of patients and the care they required" ranged from strongly agree to strongly disagree. Self-reported RN ratings were collected approximately 4 hours into each work shift. Responses from the RNs were transcribed next to nurses' names on the unit's paper-based daily assignment sheet. A light duty nurse assisted the charge nurse with data collection on some day shifts. Only charge nurses collected ratings from RNs during the night shift.
Self-reported RN ratings of assignment appropriateness were transcribed into an electronic format by light duty RNs who received data entry training from an experienced nurse informaticist. Data entry quality assurance was achieved through comparison of a sample of electronic and paper-based RN ratings by the informaticist who provided training in data entry procedures.
Data Analysis
Electronic health record-generated work intensity scores and RN self-reported ratings of appropriateness of patient assignment were analyzed at the patient care unit level, as each unit provides care to a different patient population. Box plots were generated to provide visualization of the range and central tendency of EHR-generated work intensity scores associated with each self-reported RN response category for each unit. Spearman rank correlation was used to assess the association between work intensity scores and self-reported RN perceptions collected on individual work shifts on each unit. Each paired EHR-generated work intensity score and self-reported RN rating were treated as an independent observation because the workload and situations experienced by an RN differ from shift to shift. In this study, missing data was assumed to occur at random and therefore not imputed.
RESULTS
A total of 2521 observations of paired RN-level EHR-generated work intensity scores and self-reported RN ratings of appropriateness of patient assignment were collected during a 12-week time period ending October 21, 2018. Across patient care units, paired RN ratings and work intensity scores were collected on 34 to 61 calendar days of 82 possible calendar days. Observation counts differ across patient care units due to variability in the number of on-duty RNs on any given calendar day and variability in the consistency of data collection practices in these live care environments. A total of 454 unique RNs are represented within 2521 self-reported RN ratings because some nurses worked multiple calendar days during the observation period. Accuracy of data entry, assessed through periodic spot checks, was greater than 95%. Observations of paired work intensity scores and self-reported RN ratings by patient care unit and work shift are summarized in Table 1.
The RN-level work intensity scores are spread over a large range of values, as indicated by large standard deviations in relationship to mean work intensity score. Across all patient care units, a majority of nurses agreed that their work assignment was appropriate. Nurse agreement that work shift patient assignment was appropriate was as follows: 75.9% strongly agree, 14.3% agree, 4.5% tend to agree, 3.1% tend to disagree, 1.5% disagree, and 0.7% strongly disagree. There was weak negative correlation (-0.09 to -0.232) between work intensity scores and self-reported RN ratings of assignment appropriateness (Table 2). This suggests that when EHR-generated work intensity scores increase, self-reported RN appropriateness ratings decrease. Correlation between work intensity scores and RN ratings is highest on intensive care units and lowest on labor and delivery, mother-baby, and medical-surgical units.
The boxplots in Figure 1 show the range of work intensity scores across RN appropriateness rating for each study unit. Across all units, work intensity point ranges overlap one or more RN rating categories.
DISCUSSION
Workload variability is a longstanding yet contemporary concern in nursing work systems. Pressures on hospital operating margins compel hospitals to provide care to more patients without increasing bed counts or payroll.12 Bedside nurses experience time and production pressure when patient demand exceeds RN capacity. Time constraints can contribute to negative patient and clinician outcomes when RNs must make decisions13 between competing time-sensitive care tasks.
Work intensity tools are designed to provide quantified information to guide resource allocation decisions and create equitable patient assignments in nursing work systems. Electronic health record-generated work intensity scores address several notable weaknesses of traditional patient classification systems,10 including dual data entry and lack of dynamic workload estimation. Electronic health record-based tools dynamically capture relevant patient and work activity information from nurses' clinical documentation and providers' orders as content is entered into a patient's record.
In the absence of a gold standard for measurement of work intensity, RN self-report of the perceived appropriateness of the patient assignment is a pragmatic and available point of comparison between RNs' lived work experience and estimated work intensity. In the current study, relationships between work intensity scores and RN perception of assignment appropriateness are undertaken prior to local customization of vendor-provided work intensity logic to provide insight into performance characteristics of the vendor-provided baseline logic in six patient care unit types. Electronic health record vendors typically allow hospitals to customize work intensity logic to reflect local work practices and unique workload factors, but customization is undertaken judiciously as this may necessitate additional effort during subsequent software upgrades to maintain site-specific logic.
The objective of a work intensity tool is to estimate the effort and time required to care for patients. Accurate estimation supports the creation of equitable work assignments and increases the likelihood that on-duty RNs will have sufficient time to execute high-quality, timely care for assigned patients. An underlying assumption in the evaluation of a work intensity tool is that as work intensity points increase, RNs will experience higher workload. As points increase, nurse perception of appropriateness of patient assignment is expected to decrease. In this study, work intensity scores are weakly correlated with RN ratings of appropriateness of patient assignment, indicating that higher work intensity scores are associated with lower RN appropriateness ratings. Correlation was strongest on intensive care units, where patients require around-the-clock intensive nursing care services and where nurses consistently care for 1 to 2 patients. Correlation was weakest on labor and delivery, where workload can be highly unpredictable, and on adult acute care, where RNs typically care for four or more patients in rooms that may not be geographically contiguous. Units with the lowest correlation may represent opportunities for custom configuration of work intensity estimation logic to reflect local practices and unique workload factors.
There is considerable overlap in RN report of patient assignment appropriateness across similar work intensity point ranges. Possible explanations for variability in RN rating across similar work intensity scores include a change in work intensity between the time point of work intensity capture and the time point of RN self-report, delayed EHR documentation that could artificially lower work intensity estimation, and factors not captured by EHR work intensity logic. Some factors that are perceived by nurses to contribute to workload14,15 may exist in software systems that may not yet be integrated with the EHR, such as skill mix, nurse call volume, care team communication, absence of secretary, and the presence or absence of flexible, supplementary RN resources. Variability may also be explained by differing RN perceptions of workload appropriateness or reluctance to provide a negative rating of assignment appropriateness to RN peers collecting this information. Previous studies have found that demonstrating the ability to handle one's patient load16 is highly valued in nursing culture, which has potential to bias subjective RN ratings toward agreement.
A weak correlation between work intensity scores and RN rating of appropriateness of assignment indicates that nurse perceptions decrease as EHR-generated work intensity scores increase. This finding suggests that EHR-generated scores provide useful information. However, a weak correlation between work intensity scores and RN perceptions of appropriateness indicates that clinical judgment is required to interpret and utilize EHR-generated work intensity scores in resource allocation and patient assignment decisions. In this way, EHR-generated work intensity scores can augment charge nurses' awareness of dynamic working conditions heuristically maintained through direct observation of the workplace.
LIMITATIONS
This study has several limitations, including assessment at a single academic medical center, which may limit generalizability of findings to other hospitals. Sources of potential bias include observation of the same RN on multiple work shifts and individual factors that may influence self-reported appropriateness ratings. Demonstrating the ability to handle assigned patient loads16 is highly valued in nursing culture and may bias subjective RN ratings toward agreement. Assessment of differences in perceptions of appropriateness of workload and assessment of the potential impact of missing data are beyond the scope of this study. Although dynamically generated work intensity scores can be exported from an EHR multiple times per shift, this study used a single observation to minimize RN time burden in an active care delivery environment. Future studies are needed to explore relationships between dynamic work intensity and RN perceptions across small (eg, hourly) timeframes and to assess whether changes in work intensity in a single nurses' patient assignment may influence perceived workload appropriateness by other on-duty RNs who may be called upon for assistance. Future studies are also needed to identify additional workplace factors that influence RN perception of appropriateness that may not yet be included EHR work intensity logic.
CONCLUSION
Across patient care units, RN self-reported ratings of appropriateness of patient care assignment decrease as EHR-generated estimates of RN work intensity increase. Electronic health record-generated work intensity scores can augment heuristically maintained knowledge of the nursing work environment to enhance decisions regarding nursing resource allocation and creation of RN patient assignments in a hospital setting. Additional research is needed to evaluate relationships between dynamically generated work intensity scores at multiple time points across work shifts and to identify additional factors that influence RN perceptions of patient assignment appropriateness that may not yet be captured in work intensity logic.
Acknowledgments
The authors would like to acknowledge Chia Wu for assistance with the collection of RN appropriateness ratings and data entry and Tina Won, Rebecca Tribken, and Petre Gurau for assistance with data entry.
References