The fourth annual validity study of the Health Education Systems, Inc. (HESI) Exit Exam was designed to examine not only the accuracy of the examination in predicting NCLEX success but also the degree of risk for failure of the licensure examination associated with specific scoring intervals. A descriptive comparative design was used to examine the data provided by schools of nursing regarding students' NCLEX outcomes in the 1999-2000 academic year. As in the 3 previous studies, the examination was found to be a highly accurate predictor of NCLEX success (98.46%). Each scoring interval was significantly different from each of the other scoring intervals (P = .001). In fact, for the combined group of registered nurse and practical nurse students, the percentage of students who failed the NCLEX more than doubled with each successively lower scoring interval. These findings provide the information faculties needed to make evidence-based decisions regarding students' risks for NCLEX failure. Additionally, frequency data were obtained from this survey regarding the use of the examination as a benchmark for progression and remediation, and these findings may also be useful to faculties that are considering establishing such programs.

A descriptive comparative design was used for analysis of year 4 findings. Data regarding E² scores were obtained from the HESI database. NCLEX outcomes and responses to a questionnaire were obtained from administrators at nursing schools that administered the E² during the academic year 1999-2000.

Instrument

Data were obtained from administrators or their designees at RN and PN schools of nursing that administered the E² in year 4. These schools were identified from the HESI database, newly designed in 2000-2001. Because of incompatible file formats, some data could not be exported from the database to the questionnaire format. Consequently, the population for this study was limited to only those schools whose data could be successfully exported. A total of 11,988 students took the E² during year 4: 10,546 RN students and 1,442 PN students. Data were exported from the database into the questionnaire format for 6,300 RN students, or 59.74% of the total RN student population, and 1,035 of the PN students, or 71.78% of the total PN population. Responses were received from administrators at 158 of the 166 (95.18%) RN programs and 31 of the 36 (86.11%) PN programs. Response rates for both the RN and the PN programs were considered to be representative of the 1999-2000 student populations and were comparable in size to the 3 previous studies' samples. Respondents consisted of 5,903 (86.81%) RN students and 897 (13.19%) PN students. Of the 5,903 RN students enrolled in 158 RN programs, 3,459 (58.60%) were enrolled in 92 associate-degree (ADN) programs, 2,346 (39.74%) were enrolled in 63 baccalaureate-degree (BSN) programs, and 98 (1.66%) were enrolled in 3 diploma programs. The 897 PN students were enrolled in 31 PN programs.

Data Collection Procedure

Schools that administered the E² received a summary analysis of their aggregate data. One of the reports contained in this summary analysis is a grouping of students' scores by scoring categories. These categories ranged from A, the highest scoring category, to H, the lowest scoring category. These scoring categories served as the basis for formulating the scoring intervals that were used for data collection and data analysis. Previous studies examined only high-scoring students, those who scored in categories A and B (90-99.99), and low-scoring students, those who scored in categories G and H (<= 69.99). Although the previous studies demonstrated a significant difference in the NCLEX success of high-scoring and low-scoring students, they did not provide data about middle-scoring students. Therefore, in this study, additional scoring intervals were designated to provide more discrimination in the middle-scoring groups, where the greatest ambiguity existed regarding the degree of risk for NCLEX failure. The scoring intervals comprised HESI scores designated as: A/B, scores from 90.00-99.99; C, scores from 85.00-89.99; D, scores from 80.00-84.99; E/F, scores from 70.00-79.99; and G/H, scores <= 69.99. A questionnaire, along with a cover letter inviting participation, was mailed to RN and PN program administrators at participating nursing schools. A list of the schools' students who took the E² for the first time was included in the mailing. The students' names and E² scores were grouped according to scoring intervals. Program administrators were asked how many students in each scoring interval failed the NCLEX. Names of the students were not needed for data entry, only the total numbers of failures within each group. Additional survey data were obtained from school administrators regarding the use of progression and remediation policies based on students' E² scores.

Data Treatment Procedures

Standard statistical methods were used to compute the accuracy of the E² in predicting NCLEX-RN and NCLEX-PN success. Predictive accuracy of the E² was calculated using the most stringent method, by examining only the NCLEX outcomes of those who were predicted to pass, which consisted of those scoring in categories A or B. The number of students scoring in category A/B who failed the NCLEX was divided by the total number of students who were predicted to pass, and subtracted from one. Chi squares were calculated to detect differences between expected and observed frequencies among NCLEX outcomes of students scoring at each of the 5 HESI scoring interval categories and among students in the different nursing programs scoring at each of the 5 scoring intervals. Descriptive data regarding progression and remediation policies were summarized using frequency distributions.

FINDINGS

Predictive Accuracy

In year 4, a total of 6,800 nursing students, 5,903 RN and 897 PN students, comprised the study sample. A total of 1,303 (22.07%) ADN students, 726 (12.30%) BSN students, and 30 (0.51%) diploma students, as well as 341 (38.02%) PN students, received HESI scores in the A/B category (90.00-99.99), indicating that they were predicted to pass the NCLEX-RN or NCLEX-PN without additional preparation. Of the 2,059 RN students who scored in the A/B category, 35 (1.70%) failed the licensure examination. In the ADN group, 21 (1.61%) of the 1,303 students who scored in the A/B category failed, 14 (1.93%) of the 726 BSN students who scored in the A/B category failed, and none (0%) of the 30 diploma students who scored in the A/B interval failed. Of the 341 PN students who scored in the A/B category, 2 (0.59%) failed the NCLEX-PN.

The predictive accuracy of the E² in year 4 was 98.30% for RN students, 99.41% for PN students, and 98.46% for all students. Results of a chi-square goodness-of-fit revealed that the year 4 predictive accuracy for all students (98.46%) was not significantly different from that of year 1 (97.41%), year 2 (96.49%), or year 3 (97.78%). Thus, as in the 3 previous years of study, the predictive accuracy of the E² was not significantly different among the years of study. Also, as was the case with the 3 previous years of study, there was no significant difference in the predictive accuracy by types of programs examined (ADN, BSN, diploma, or PN).

Predictive Accuracy by Scoring Intervals

Previous studies examined only the outcomes of the high-scoring and low-scoring E² students and found that significantly more of the low-scoring students failed NCLEX than did the high-scoring students ([chi] = 571.401, P = .001). Year 4 not only examined high-scoring and low-scoring students but also compared NCLEX outcomes for the 5 HESI scoring intervals. A chi-square analysis of the students' scores for each of the 5 scoring intervals revealed significant differences ([chi] = 618.816, P = .001) among scoring intervals of RN students' scores as well as significant differences ([chi] = 211.624, P = .001) among scoring intervals of PN students' scores.

Students' outcomes were compared by program type for each of the scoring intervals, eg, A/B interval scores of ADN students were compared with A/B interval scores of BSN students. Scores at all intervals were compared for each of the 4 programs until an analysis of all possible combinations of programs was completed. No significant difference was found in the predictive accuracy of the E² among programs (ADN, BSN, diploma, or PN) for students scoring at like levels in each of the 5 scoring intervals. In other words, significant differences in NCLEX pass rates were noted for each scoring interval, but no significant difference was found among different programs when they were compared at like intervals. However, when aggregate data regarding NCLEX outcomes for all RN programs were compared with aggregate data for all PN programs at each of the 5 scoring intervals, a significant difference was found between RN and PN students in the lowest scoring categories only, the E/F (70.00-79.99) and G/H (<= 69.99) categories. Significantly more RN students scoring in the E/F ([chi] = 34.545, P = .001) and G/H ([chi] = 8.926, P = .01) categories passed the NCLEX, as compared to PN students scoring in these two categories.

Use of the E

NCLEX results and frequency data regarding remediation and progression policies were obtained from the questionnaires mailed to participating schools. Administrators were asked if their schools had implemented progression or remediation policies based on E² scores. Of the 158 participating RN programs, 149 (94.30%) responded to the question regarding progression policy adoption and 45 (30.20%) of these schools tied progression to a minimally acceptable E² score for students to graduate or to take the licensing examination. Of the 31 participating PN programs, 5 (16.13%) indicated that they used E² scores as a benchmark for remediation, and these same 5 schools tied progression to a minimally acceptable E² score for students to graduate or to take the licensing examination. Thirty-four (75.56%) of the 45 RN schools and 2 (40.00%) of the 5 PN schools reported using 85 as the minimally acceptable HESI score required for progression. Only 2 (4.44%) of the 45 RN schools reported using an E² score below 85, with 77 being the lowest reported score used as a benchmark, whereas 6 (13.33%) indicated that they used an E² score higher than 85, with 90 being the highest reported score used as a benchmark. Three of the RN schools with established progression policies did not provide information about their minimally acceptable scores. Of the 5 PN schools, 2 (40.00%) reported using an E² score below 85, and both used a score of 75, whereas only 1 (20.00%) indicated using an E² score higher than 85 as a benchmark, and that higher score was 86.

Remediation Strategies Used to Improve E

Certain limitations of the research methodology and analytic techniques must be considered when reviewing the findings of this study. Incorporation of more sophisticated computerized tracking techniques into the data acquisition process rendered some subjects' scores inaccessible for this study. Although approximately 50% more E²s were administered in year 4 than in year 3, the year 4 sample size was only slightly larger than that of year 3. Additionally, the research design lacked control of intervening factors that may have occurred between E² administration and administration of the NCLEX-RN or NCLEX-PN. Such factors could have influenced the predictive accuracy of the E².

Despite using the most stringent method for calculating predictive accuracy, the E² continued to demonstrate a high degree of accuracy in predicting NCLEX success (98.46%). For the fourth consecutive year, the predictive accuracy of the E² was not significantly different among the programs tested (ADN, BSN, diploma, or PN), indicating that the E² is an effective predictor of NCLEX success for all types of nursing programs. Additionally, the predictive accuracy of the E² in year 4 was not significantly different from the predictive accuracy of the E² in the 3 prior years of study, indicating that, based on the aggregate data collected from 19,554 subjects during 4 consecutive years, the E² is a highly accurate predictor of NCLEX success.

In year 4, an analysis of students' E² performances by specific scoring intervals revealed a consistent pattern. The percentage of students who failed the NCLEX significantly increased with each successive drop in scoring interval, creating a stepwise pattern of progressively higher percentages of subjects failing the NCLEX-RN. The pattern exhibited by the PN students was similar to that of the RN students, but the degree of risk for failure of the NCLEX-PN was more pronounced in the lowest two scoring intervals. This pattern could be a reflection of the lack of remediation reported by the PN schools because only 16.13% of these schools required remediation for those students who obtained low E² scores.

The finding of no significant differences in scoring intervals among RN program types indicates that inferences regarding risk for NCLEX-RN failure are the same regardless of program type. However, the finding that significantly more RN students who scored in the two lowest scoring intervals passed the NCLEX than did PN students who scored in these same categories is likely attributable to PN students usually taking the E² just before completion of their programs and not having the opportunity to remediate. The percentage of failures at each scoring interval provides the data necessary to make inferences regarding the degree of risk for NCLEX failure associated with each scoring interval.

Only 28.19% of the RN schools and 16.13% of the PN schools reported having implemented progression policies based on E² scores. However, since these data were collected, HESI reports that many more schools have implemented such policies or are considering implementing such policies. Although data from this study indicated that most schools with established progression policies required a minimal HESI score of 85 for progression, such decisions may, of necessity, be influenced by class size. For example, schools that graduate 10 or fewer students can ill afford to have more than one student fail the NCLEX. Therefore, such schools may elect to set higher benchmarks than schools that graduate larger numbers. Finally, in reviewing data regarding the effectiveness of remediation programs, all schools that had implemented such programs found them effective and reported that most of the remediated students passed the NCLEX on their first attempts. Of those schools that responded to the question about success of second-time candidates to the licensing examination, 75% of the RN schools and 100% of the PN schools indicated that these candidates were successful on their second attempts. It appears from these data, therefore, that implementation of such policies assists the at-risk student to become licensed and helps to ameliorate the problem of graduates who never pass the NCLEX.

RECOMMENDATIONS

The recommendations of the year 3 study were considered in the design of this year 4 study. Findings indicated that schools of nursing are implementing policies that use E² scores as benchmarks for progression and remediation. To better define the concept of remediation, administrators were asked to describe the programs they had implemented. Frequency data indicated that these progression and remediation programs were effective in increasing NCLEX pass rates.

As in the previously published studies, the E² was determined to be highly accurate (98.46%) in predicting NCLEX success. These findings regarding the validity of the E² will assist nurse educators in explaining to students their risks for NCLEX failure and provide the empiric data that may be useful in convincing students of the necessity to remediate before taking the NCLEX.

The E² has consistently been identified as a highly accurate predictor of NCLEX outcomes. Thus, many nursing faculties have implemented policies that use E² scores as progression and remediation benchmarks. When establishing such a policy, it is the school's responsibility to designate the required E² score for progression. Results of this study regarding the degree of risk associated with various scoring intervals provide the evidenced-based support that faculties need when determining specific E² scores to use as benchmarks for progression and remediation. Additionally, these findings may assist faculties who are debating the value of adopting progression and remediation policies.

Acknowledgments

The authors thank Susan Morrison, PhD, RN, President, Health Education Systems, Inc (HESI), for granting permission to use the HESI database, and the program administrators for their participation in this study. They also wish to recognize Donna Boyd for her editorial assistance with preparation of the manuscript.

REFERENCES