Authors

  1. Reiter, Margaret Ann PhD, RN
  2. Young, Anne EdD, RN
  3. Adamson, Carolyn PhD, RN

Abstract

Predicting workplace success is becoming increasingly important because the cost of orienting new nurse graduates is increasing at such an alarming rate. This study assessed the effectiveness of the HESI Exit Exam in measuring entry-level competencies of novice nurses. Findings indicate that the HESI Exit Exam was an effective predictor of workplace competency for new graduates (N = 108) assigned to acute care and critical care units in a large, tertiary care hospital.

 

Article Content

Sixty percent of all new graduate registered nurses (RNs) seek employment opportunities in hospitals throughout the United States.1 The cost incurred by healthcare institutions to orient these new graduates and media focus on mistakes made in acute care settings has increased the pressure to provide a competent nursing workforce.2 This workforce includes new graduate nurses who have limited clinical experience and are likely to have no previous employment history in nursing.

 

Numerous studies have examined the best methods for preparing new graduates for the work environment and, in particular, to function as nurses in the acute care setting.3-6 A plethora of orientation programs has been developed to meet the needs of novice nurses. These multifaceted residency, internship, or preceptorship programs last from 4 to 12 months and are designed to provide new graduates with supervised hands-on clinical experience.7 Positive preceptoring relationships and adequate support systems during orientation favorably impacted the new graduates' feelings of acceptance in the workplace.8 Healthcare institutions continue to support the use of preceptors to conduct orientation for newly hired nurses. Many of those who serve as preceptors have as little as 1 year of experience in the same setting as the new employee.9

 

During hospital orientation, employers are mandated to assess the initial competence of all employees. 10 New graduate nurses are evaluated on several skill sets, including critical thinking, prioritization, interpersonal communication, application of nursing process, and medication administration. This assessment process requires a significant investment of resources by the employing institution. The cost of assessing and training a new graduate for employment in acute care hospitals has increased exponentially since 1987, when Hughes4 reported that the cost of orientation for a new graduate was between $1,000 and $3,000. In 1992, Burns and Hutchens11 reported that the orientation cost for each new graduate was between $7,800 and $12,000. In 1995, Messmer et al12 estimated that the cost of orienting a new nurse was between $20,000 and $50,000.

 

In 2004, Lindy and Reiter13 reported that the cost to orient 42 new graduates to a hospital was $611,455, or $14,558 each. However, this cost did not include expenses related to turnover. Beecroft et al14 reported that the cost of replacing an RN is between 75% and 125% of the RN's annual salary because of costs related to recruitment, overtime compensation to ensure coverage, orientation of replacement employees, lost productivity, and customer satisfaction. Because 20% of the 42 new graduates in the Lindy and Reiter study were not retained, the estimated cost to orient each of these 42 new graduates was between $39,000 and $65,000.

 

This increase in the cost for orientation is alarming. Hospital administrators and nurse managers need evaluation measures to assist them in assessing candidates so that the best possible candidates are selected for employment. Because orientation is a significant budgetary item, nurse managers and administrators need an objective measure to assist them in deciding who should receive this sizable investment.

 

Although predictors of success in completing basic nursing degree programs and achieving a passing score on the National Council Licensure Examination for Registered Nurses (NCLEX-RN) have been extensively examined, predictors of success within the first year of employment after graduation have not been as clearly identified. Managers usually make hiring decisions about new graduates based on their grade point averages and recommendations solicited from their faculty. Jones et al5 reported that a grade point average of 3.0 and a successful interview are the documented criteria for workplace success. Decisions regarding employment are typically based solely on the subjective, professional judgment of the nurse manager rather than on an objective evaluation measure.

 

Purpose of the Study

The hospital nursing education staff at a 900-bed, tertiary care hospital located in a major medical center in southeast Texas recognized that the many academic institutions from which their applicant pool graduated required senior students to successfully complete the Health Education Systems, Inc, Exit Exam (HESI E2) before graduation. A review of the literature revealed that the HESI E2 is a highly accurate predictor of NCLEX-RN examination success,15-21 and like the NCLEX-RN examination, it measured entry-level competence.

 

To determine if this tool might also be an effective measure of entry-level workplace competency, a pilot study was conducted with 41 new graduate nurses enrolled in the hospital's critical care internship program. Findings of this pilot study concluded that the HESI E2 was predictive of new graduate success in this program. Based on the findings of the pilot study, this study was designed to compare traditional measures of entry-level RN competency with the HESI E2 using a larger sample of new graduate nurses assigned to both critical care units and acute care units within the hospital.

 

Methodology

A descriptive, correlational design was used to assess the relationship between HESI E2 scores and workplace competencies of new graduate nurses (N = 108) assigned to 16 acute care units and 10 critical care units. The sample consisted of diploma, associate degree, and baccalaureate nursing program graduates. The HESI E2 was administered to these subjects before graduation and before their employment in entry-level staff nurse positions. Scores from 5 instruments provided the data for this study: (1) the HESI E2, administered during the final semester or quarter of the nursing school curriculum, (2) NCLEX-RN examination, reported by the Texas Board of Nurse Examiners, (3) the Performance Management Systems, Inc (PMSI), (4) the Medication Administration Safety Test (MAST), and (5) a 90-day performance appraisal.

 

The HESI E2 is a 150-item computerized examination that is based on the NCLEX-RN examination test blueprint. The examination is used throughout the United States and internationally to predict licensure success and to identify individual candidates' specific remediation needs before taking the licensure examination. Numerous research studies have examined the reliability and validity of the HESI E2, and findings indicate that it is between 96.36% and 100% accurate in predicting NCLEX-RN examination success.15-21 The licensure examination, the NCLEX-RN examination, is used to measure entry-level competency and is administered by the boards of nursing in all 50 states, the District of Columbia, and 5 US territories. It is a computer-adaptive test, and scores are reported as passed or failed.22

 

The PMSI competency evaluation is used by more than 500 hospitals nationwide to measure critical thinking.23 It was administered to all new graduates hired by the study hospital for entry-level positions. A scoring methodology was assigned from 0 to 130.

 

The MAST is a computerized examination that is used nationally to evaluate competency in calculating dosages of oral medications, intramuscular medications, and intravenous preparations and in making weight-based dosage adjustments. Correct responses to questions in each of these 4 sections are worth 25 points, with a maximum possible score of 100.

 

The 90-day performance appraisal was constructed by nursing administrators at the study hospital. It was completed by the subjects' immediate supervisors, and a Likert-type scale was used to subjectively rate the new employees' performance. Scores range from 0, indicating failure to achieve expectations, to 20, indicating consistently exceeds expectations.

 

Findings

The sample consisted of 99 (91.67%) women and 9 (8.33%) men, who graduated from baccalaureate (n = 91; 84.26%), associate degree (n = 16; 14.81%), and diploma nursing programs (n = 1; 0.93%) and were administered the HESI E2 before graduation. These new graduate employees were assigned to either acute care units (n = 58; 53.70%) or critical care units (n = 50; 46.30%).

 

As students, the nurses' scores on their first testing with the HESI E2 ranged from 60.00 to 99.99, with a mean score of 87.04,a (SD, 10.98; median, 88.81). It was beyond the scope of this study to investigate the HESI E2 scores of those who took multiple, parallel forms of the examination. Those who scored 90.00 and above on the HESI E2 were predicted to pass the NCLEX-RN examination on their first attempt.24 Of the 108 nurses, 49 scored 90.00 and above on the HESI E2, and all 49 (100%) who were predicted to pass were successful on their first attempt at taking the NCLEX-RN examination. When taking the NCLEX-RN examination for the first time, 103 (95.37%) of the 108 nurses were successful. Only 5 nurses (4.63%) failed the NCLEX-RN examination on their first attempt, and all 5 scored below 79.99 on the HESI E2.

 

Scores on the PMSI competency assessment ranged from 45.38 to 109.00 (possible range, 0-130), with a mean score of 75.10 (SD, 13.67; median, 72.65). Scores on the MAST ranged from 52.00 to 100.00 (possible range, 0-100), with a mean score of 79.16 (SD, 11.91; median, 77.00). Scores on the 90-day performance appraisal ranged from 9.00 to 17.00 (possible range, 0-20), with a mean score of 11.37 (SD, 1.86; median, 10.62). Table 1 describes the data obtained from these 5 instruments. Using a [chi]2 test of independence, HESI E2 scoring groups were found to be significantly related to NCLEX-RN examination outcomes ([chi]24 = 13.49, P < .01).

  
Table 1 - Click to enlarge in new windowTable 1. Mean, Median, and SD for HESI E

To evaluate the relationship of HESI E2 scores with the PMSI competency evaluation, the MAST, and the 90-day performance appraisal, all scores were converted to z scores, and Pearson correlation coefficients were run between the variables. Table 2 describes these correlations.

  
Table 2 - Click to enlarge in new windowTable 2. Correlations of HESI E

HESI E2 scores and PMSI competency assessment scores were positively correlated (r [106] = 0.26, P < .01); r2 = 0.07, which indicates that 7% of the variance in PMSI scores was associated with HESI E2 scores. HESI E2 and MAST scores were also positively correlated (r [106] = 0.22, P < .05); r2 = 0.05, which indicates that 5% of the variance in MAST scores was associated with the HESI E2 scores. HESI E2 scores and the 90-day performance appraisal scores were positively, but not significantly, correlated. Table 2 describes these correlations.

 

Discussion of Findings

The findings of this study validated previously published research findings that described the HESI E2 as highly accurate in predicting NCLEX-RN examination success. For this study group, the HESI E2 was 100% accurate. All 5 subjects who failed the NCLEX-RN examination on their first attempt scored below 79.99 on the HESI E2.

 

HESI E2 scores were positively correlated with NCLEX-RN examination outcomes, as well as with 2 measures of new employee competence, the PMSI competency assessment scores that measure critical thinking and priority communication and the MAST scores that measure competency in calculating and administering medications. The association of HESI E2 scores with the new graduates' 90-day performance appraisal scores was not statistically significant. This finding is possibly related to the fact that the HESI E2 is an objective measurement of entry-level competence, whereas the 90-day performance appraisal is a subjective assessment of new graduates' performance by their immediate supervisors.

 

The findings of this study provide a new and reliable quantitative employment selection criterion for nurse managers beyond the grade point average and interview processes suggested by Jones et al.5 Selection of new graduate nurses with higher HESI E2 scores allows the nurse manager to focus on competency as a means of ensuring patient safety. The HESI E2 is designed to evaluate the new graduate nurses' ability to function as entry-level nursing professionals. The findings of this study provide nurse managers with a measurement tool that enables them to evaluate the candidates' ability to think critically within the discipline of nursing, as evidenced by their HESI E2 scores. The relationship of the HESI E2 with widely used assessments of employment competency may provide a common ground for evaluating new graduates' abilities before employment. The HESI E2 may address the need for ensuring a safe, competent workforce.

 

Nursing students eager to secure employment in institutions with competitive residency programs are willing to share their HESI E2 scores. Chief nursing officers can save on both orientation and replacement dollars. The retention rate for new graduate nurses during this study was 98% after 1 year.

 

Conclusion

The use of HESI E2 scores as a predictor of employment success by nurse managers and educators was validated by this study. The application of academic testing results as a screening tool for employment was successful at the study hospital. HESI E2 scores were not only effective predictors of new graduates' NCLEX-RN examination outcomes, but also effective measures of new graduate nurses' workplace competency. As such, HESI E2 scores are potentially useful determinants of the new graduate nurses' ability to successfully assume entry-level positions within the practice setting.

 

The strong retention rate associated with this study may be related to the focus on clinical orientation as opposed to classroom orientation. Chief nursing officers might consider accepting HESI E2 scores in lieu of standardized competency assessments completed during the orientation process.

 

References

 

1. The Advisory Board Company. Reversing the Flight of Talent. Vol II. Washington, DC: Author; 2000. http://www.advisoryboardcompany.com. Accessed October 16, 2006. [Context Link]

 

2. Spath P. Can you hear me now? Hosp Health Netw. 2003;77(12):36-38. [Context Link]

 

3. Dunn SV. Orientation: the transition from novice to competent critical care nurse. Crit Care Nurs Q. 1992;15(1):69-77. [Context Link]

 

4. Hughes L. Employment of new graduates: implications for critical nursing practice. Focus Crit Care. 1987;14(4):9-15. [Context Link]

 

5. Jones TL, Mims BC, Luecke LE. Two successful models for preparing competent critical care nurses. Crit Care Educ. 2001;13(1):35-45. [Context Link]

 

6. Grossman S, Campbell C, Riley B. Assessment of clinical decision-making: ability of critical care nurses. Dimens Crit Care Nurse. 1996;15(5):272-279. [Context Link]

 

7. Herdrich B, Lindsay A. Nurse residency programs: redesigning the transition into practice. J Nurs Staff Dev. 2006;22(2):55-62. [Context Link]

 

8. Boyle DK, Popkess-Vawter S, Taunton RL. Socialization of new graduate nurses in critical care. Heart Lung. 1996;25(2):141-154. [Context Link]

 

9. Brown PL. Graduate nurses: what do they expect? Kans Nurse. 1999;74(5):4-5. [Context Link]

 

10. Joint Commission on Accreditation of Healthcare Organizations. Standards of Care for Hospitals. Oakbrook Terrace, IL: Joint Commission Resources; 2004. [Context Link]

 

11. Burns SM, Hutchens AL. New graduates in critical care: how long do they stay? Crit Care Nurse. 1992;12(8):74-79. [Context Link]

 

12. Messmer PR, Abelleira A, Erb PS. Code 50: an orientation matrix to track orientation cost. J Nurs Staff Dev. 1995;11(5):261-264. [Context Link]

 

13. Lindy CN, Reiter P. The financial impact of staff development. J Contin Educ Nurs. 2006;37(3):121-127. [Context Link]

 

14. Beecroft P, Kunzman L, Krozek C. RN internship: outcomes of a one-year pilot program. J Nurs Adm. 2001;31(12):575-582. [Context Link]

 

15. Lauchner K, Newman M, Britt R. Predicting licensure success with a computerized comprehensive nursing exam: the HESI Exit Exam. Comput Nurs. 1999;17(3):120-125. [Context Link]

 

16. Newman M, Britt R, Lauchner K. Predictive accuracy of the HESI Exit Exam: a follow-up study. Comput Nurs. 2000; 18(3):132-136. [Context Link]

 

17. Nibert A, Young A. A third study predicting NCLEX success with the HESI Exit Exam. Comput Nurs. 2001;19(4):172-178. [Context Link]

 

18. Nibert A, Young A, Adamson C. Predicting NCLEX success with the HESI Exit Exam: fourth annual validity study. Comput Inform Nurs. 2002;20(6):261-267. [Context Link]

 

19. Daley LK, Kirkpatrick BL, Frazier SK, Chung ML, Moser DK. Predictors of NCLEX-RN success in a baccalaureate nursing program as a foundation for remediation. J Nurs Educ. 2003;42(9):390-398. [Context Link]

 

20. Morrison S, Adamson C, Nibert A, Hsia S. HESI exams: an overview of reliability and validity. Comput Inform Nurs. 2004;22(4):220-226. [Context Link]

 

21. Lewis C. Predictive accuracy of the HESI Exit Exam on NCLEX-RN pass rates and effects of progression policies on nursing student Exit Exam scores. Dissertation Abstracts International. 2005;66(11):154B (UMI No. 3195986). [Context Link]

 

22. National Council of State Boards of Nursing. Candidate Bulletin. National Council of State Boards of Nursing. http://www.ncsbn.org/resources/complimentarynocostexam.asp. Accessed October 16, 2006. [Context Link]

 

23. Performance Management Systems, Inc. http://www.pmsipbds.com. Accessed October 16, 2006. [Context Link]

 

24. Health Education Systems, Inc. http://www.hesitest.com. Accessed October 16, 2006. [Context Link]

 

aIn 2004, HESI scores were converted to three-digit scores by multiplying the score by 10; for example, a HESI score of 87.04 is now expressed as 870. [Context Link]