Providing competent professional nurses through first-time success on the National Council Licensure Examination for Registered Nurses (NCLEX-RN) has always been a primary objective of nursing programs. State boards of nursing and national accrediting organizations require acceptable NCLEX pass rates for program approval. Faculty and students share the goals of licensure and launching a nursing career. A consistent record of NCLEX success also attracts highly qualified new students to a program. In light of the nursing shortage, educational programs face unprecedented pressure to expedite entry of nurses into the work force via first-time success on the NCLEX-RN.
Over the past 2 decades, many studies have attempted to identify academic and nonacademic predictors of NCLEX-RN performance. Despite researchers' best efforts, development of a comprehensive and consistent model of prediction has been complicated by the ever-changing NCLEX-RN test plan, question format, passing standard, and the enigmatic interaction of demographic, academic, and psychosocial variables. While most researchers agree that predicting success on the NCLEX-RN is more accurate than predicting failure,1 the steady decline in first-time NCLEX-RN pass rates of baccalaureate-prepared candidates, from 89% in 1995 to 84% in 2000, has been troubling.2 Although the 2003 pass rate improved to 87%, recent modifications in the NCLEX-RN question format, test plan, and increased passing standard cause renewed concern. The potentially adverse effects of these changes on first-time NCLEX pass rates challenge researchers to venture beyond prediction to investigate the effectiveness of various strategies on performance.
Method and Inclusion Criteria
A literature search was performed using CINAHL, Medline, and ProQuest databases from 1984 to 2004 and key words such as "NCLEX success," "NCLEX interventions," and "NCLEX performance." "Nursing student retention" and "nursing student attrition" were also used as key words so that articles describing an intervention, or using an intervention as an independent or dependent variable, could be retrieved. Reference lists from each retrieved article were also examined for pertinent studies that were not uncovered in the initial search.
Inclusion criteria for the review were (a) intervention studies involving baccalaureate-prepared, first-degree nursing students in the past 20 years; (b) randomized controlled trials or quasi-experimental designs with some measurable outcome of NCLEX success; and (c) studies conducted and published in peer-reviewed, US-based, nursing journals. Dissertation Abstracts Online was also searched for unpublished dissertations in the past 20 years, but ultimately was excluded from the review because little was found. Exclusion criteria included (a) studies with retention or program success, however defined, as the only outcome variable and (b) studies that examined strategies exclusively for minority or English as second language (ESL) populations. While these studies are nonetheless very important, they were excluded from this more general review. The search yielded 8 relevant studies.
The most frequent source of articles was the Journal of Nursing Education with 4 studies on the subject.3-6 Two reports were published in Nurse Educator7,8 and 1 each in CIN: Computers, Informatics, Nursing (formerly Computers in Nursing9) and Nursing and Health Care.10 The most recent study meeting the inclusion criteria was published in 2001 while the least current was published in 1987.
Another group of studies discussed separately used the HESI Exit Exam to set benchmarks for progression and remediation. Five studies11-15 focused on the predictive value of the HESI for first-time NCLEX pass rates, representing a recent trend to identify at-risk students on the basis of performance on commercially available computerized examinations. Four of these studies appeared in CIN: Computers, Informatics, Nursing and 1 in Nurse Educator.
Review of Intervention Studies
Studies meeting the inclusion criteria are reviewed below and summarized in Table 1.
Outcome Variables
Seven studies examined NCLEX success as the only outcome variable,3-5,7-10 whereas one focused on attrition as well.6 Improvement in NCLEX pass rate was the most frequently reported finding although there was wide variation in how this was measured. Some studies compared the NCLEX pass rates for 2 consecutive years, the first year with no intervention and the second year with intervention. Others had a 2-year interval between comparisons. No study compared the effectiveness of interventions beyond 2 years. While improved NCLEX pass rates were reported, tests for statistical significance were infrequent.
Sample Size
Three of the 7 studies5,7,10 did not disclose the number of participants. In the 5 remaining studies, sample size ranged from 22 to 450. Interestingly, the study by Ross et al8 with the largest number of participants (n = 230) was designated a "pilot" study.
Types of Interventions and Target Groups
Interventions included (1) study groups or test-coaching approaches that addressed content areas and/or test-taking skills; (2) support groups to offer intimate interaction, guidance, and feedback combined with encouraging the development of problem-solving skills; (3) individualized study plans based on results of a diagnostic practice examination and/or assessment; (4) comprehensive advisement and development program; (5) comprehensive academic success plan; and (6) computer-based testing.
Five of the studies targeted only at-risk students.3-6,10 In 2 studies by the same authors, at-risk was defined as a grade-point average (GPA) at or below a 2.40, a Mosby Assess Test score below the 20th percentile, and/or a diagnosed learning disability.3,4 In Brown's study, at-risk students were juniors with an 80 to 84 average in the 1st semester nursing course,6 while in the study of Baradell et al, at-risk students were identified on the basis of SAT score, entry GPA, and GPA in junior year nursing courses.5 Ross et al identified at-risk students as those who had experienced any failure in upperdivision nursing courses.8
Wolahan and Wieczorek10 required that all students achieve a score of 75% on the NLN Diagnostic Readiness Test (NLN-DRT). Those who did not were identified as at-risk and required to participate in a comprehensive academic success plan.
Conversely, all students had the opportunity to engage in the entire intervention plan in 2 of the studies.7,8 In Eason and Woolard's study, all students took the NLN-DRT, were given a report card, were required to attend weekly sessions, and had the option of using discovery laboratories and other resource materials.7
Ross et al offered students the opportunity to take a practice examination, participate in an NCLEX review course, and use other resources.8 At-risk students were also given the option of self-assessment and development of an individualized study plan with progress monitored by faculty advisor. None of the 230 students, including the 46 at-risk students, completed the study plan and only 21% of the total sample attended the NCLEX review course.
Ashley and O'Neill found that more than half of the students who were offered a content review did not participate.3,4 In Reising's study, all students participated in a computer-based testing intervention embedded in selected courses in the curriculum.9
Effectiveness of Interventions
The results were as varied as the interventions themselves. In Ashley and O'Neill's study, the pass rate for at-risk students who received 10 two-hour sessions on test-taking/skill development was 93% compared to 50% in the control group of at-risk students.4 With a larger sample, using the same definition of at-risk students and the same intervention, Ashley and O'Neill reported an NCLEX pass rate of 86% for the intervention group compared with a 50% pass rate for the control group.3 In Brown's examination of the effects of a support group, the pass rate for the treatment group was 89% compared to 62% in the control group.6
Ross et al offered a study plan, NCLEX practice examination, and review course.8 While 85% of the 46 at-risk students passed NCLEX compared to 90% of the 184 not at-risk students, the most notable finding was low participation in the optional strategies, particularly among the at-risk students. Using a 2-pronged approach of academic development and advisement, Baradell et al reported that NCLEX pass rates for at-risk students increased from 44% in 1986 to 71% in 1988.5
Eason and Woolard noted a 27% increase in NCLEX success for 1 class who received a report card intervention as compared to a previous class who received no intervention. Unfortunately only group differences were reported rather than actual pass rates for each class.7 Wolahan and Wieczorek developed a comprehensive Nursing Education Success plan involving curriculum revision, advisement, group study sessions, seminars, NCLEX review books and courses, and tutoring for at-risk students, and found an increase in the pass rate from 37% in 1989 to 94% in 1990.10
In Reising's study, there was no significant difference in NCLEX pass rates of students who used a commercially available computer software testing intervention and those who did not.9 However, the author postulated that the 1998 increased passing standard caused a decline in pass rates during that period and influenced the study results.
Several other studies, not meeting inclusion criteria but worthy of mention, described NCLEX interventions but did not report an outcome measure. Most included a multifaceted approach and faculty involvement. Stark et al described empowering strategies including individual self-assessment and planning to assist senior-level students in recognizing strengths and areas for improvement.16 In a study of at-risk students, Williams and Bryant detailed a study plan of completing questions from NCLEX review publications, using computer programs with periodic faculty supervision and discussing progress.17 Kaufman et al described a course for NCLEX preparation that involved repeated tests of various clinically oriented content areas, approaches to improve question analysis, resource guides, informal discussion sessions, and optional 1-to-1 counseling.18 As noted in other studies where interventions were not mandatory, only 15% of the students participated in the optional counseling.
Siktberg and Dillard discussed modifications to admission and progression policies, course changes, and implementation of a computerized NCLEX-RN review course. As a result, an increase in the NCLEX pass rate above the national mean for 6 consecutive years was reported.19 Focusing on psychological preparation, Houchin administered mock NCLEXRN examinations to reduce student anxiety by familiarizing them with the format.20
Interventions With Diverse Groups
Studies of interventions with diverse student groups are sparse with recent studies virtually nonexistent. With a class of 8 African American associate-degree students, Frierson et al used a 3-pronged approach, consisting of instructions in effective test-taking, participation in learning teams, and follow-up activities conducted by faculty to reinforce the first 2 components. They reported a 100% pass rate compared to a 65% pass rate for the previous 4 years combined.21
Most recently, Cunningham et al implemented tutoring, NCLEX question practice, a comprehensive review course, and psychosocial support with a group of ESL students. No results were reported but the authors noted that such strategies were general solutions and that interventions should be adapted to the individual needs of the ESL student. They also emphasized early identification and intervention for struggling ESL students.22
Benchmarking Using Commercially Available Computer Testing
A final group of studies used the HESI Exit Exam as both a measure of preparedness to take the NCLEX-RN and a benchmark for remediation.13 The HESI Exit Exam, a comprehensive computerized examination, uses a proprietary mathematical model to give immediate feedback to students in the form of a probability score.17 HESI examination scores have been found to be highly predictive of NCLEX success with results in the range of 96% to 99%.12,14,15,19 Morrison et al summarized an initiative by 5 schools, including associate and baccalaureate degree programs, where progression and remediation policies were based on HESI Exit Exam scores. While the results indicated a 9% to 41% improvement in pass rates within 2 years of implementing new policies, comparison of results was difficult because of variability across programs in both the score necessitating remediation and in the remediation provided.23
Discussion
After analysis of the relevant research published in peer-reviewed US nursing journals, only 8 were found that measured the effect of an intervention on NCLEX-RN success in baccalaureate graduates. The majority of articles merely described methods to enhance first-time NCLEX pass rates and/or promote retention. Despite the paucity of intervention studies, this review brings several concerns to the forefront including those related to sample, design, methodology, measurement and statistical analysis, other important variables not studied, and limited faculty resources.
Issues related to sample included size, lack of randomization, and variability in determining at-risk status. Reporting of sample size was sporadic, with no sample size reported for 3 of 8 studies and the remaining with wide variation from 22 to 450 participants. Clearly, more research with larger groups would improve the ability to draw conclusions about the effectiveness of particular interventions. Randomization was also lacking in the majority of the studies. While the most rigorous research generally involves random assignment to treatment or control groups, NCLEX intervention studies were conducted in 2 ways: comparing the first-time NCLEX pass rates for at-risk students to not-at-risk students within a single cohort where the intervention was offered to all students on an optional basis, or comparing 2 cohorts of at-risk students, 1 of which participated in an intervention.
Although separation into groups by at-risk status is one way of creating groups, it is hardly random. Using this methodology could be dangerous given the implications of labeling a student as at-risk or the possibility of placing students in an incorrect group. Comparison of existing studies was further complicated by the variability in determining at-risk status. No uniformity existed as to what was included in the calculation of at-risk status and when the at-risk determination was made. Testing groups for equivalency on academic predictors prior to an intervention is one method to improve rigor and was reported in 3 studies.3,4,6
Flaws in design introduced confounding variables that influenced the ability to determine the effectiveness of the intervention. Most were confounded by variation in participation since the interventions were optional. Of the 4 studies reporting participation rates, many disclosed compromised participation when the intervention was optional.3,4,8,9 Not surprisingly, students indicated that class or work schedules precluded them from taking advantage of the interventions. Timing of interventions also varied from early in the junior year to just prior to graduation raising questions about causation (Was it the intervention, its proximity to NCLEX testing, or both?). Finally, there was considerable difference in the duration of interventions from 1 hour to an unspecified length of individual advisement again creating confounding factors.
Lack of standardization of the interventions evaluated by various researchers raised methodologic concerns. Although group study and development of individualized plans were the most commonly studied, their effectiveness was difficult to determine since they varied in breadth. Some studies outlined extensive changes in curriculum, academic advising, admission and progression policies, along with structured NCLEX review and other tutoring programs, while others were more limited in scope. Programs with more stringent admission and progression policies may effectively "weed out" students who would fail NCLEX before any intervention is implemented. Some standardization or consistency in interventions would improve understanding of the phenomena under study.
While improvement in NCLEX pass rate was a reasonable research outcome, most measurement was limited to a comparison of 1 year of graduates with another. Evaluating the effectiveness of interventions with multiple cohorts would provide better information about the usefulness of a particular strategy. Also, most researchers failed to report any analysis for statistically significant improvement in pass rates, noting only percentage of students passing NCLEX. Interventions were then declared effective on the basis of improvement in NCLEX pass rates that could be attributed to chance. However, statistical significance is meaningless without practical significance. Thus an improvement in pass rates from 25% to 50% is statistically significant, but the practical significance is that 50% of the students are still failing. In some instances, positive feedback from students and/or faculty was used as justification for continuing interventions, whether NCLEX pass rates improved or not. Lack of rigor in measurement and statistical analysis could lead programs to use interventions whose effectiveness is largely unknown.
Previous research has demonstrated that nonacademic factors, including family crises, life events, illness, test anxiety, and lack of confidence, can negatively impact NCLEX performance and often are not considered when assessing risk for NCLEX failure.20 None of the studies included nonacademic variables that might have influenced participation in interventions, NCLEX performance, or their interaction to create an especially high-risk status. A secondary, but nonetheless important, finding was the lack of intervention studies focused on diverse populations, including students with English as a second language and those enrolled in nursing as a second degree.
Finally, limited faculty resources to implement and coordinate the prescribed interventions were a significant concern. Only 2 studies reported having a faculty member with release time to oversee an intervention project and/or funding allocated for faculty development and implementation of an intervention.8,10 While the assignment of a faculty member to implement any intervention is most desirable, the worsening faculty shortage makes this unlikely.
Conclusion
While this review was not intended to be exhaustive, the studies meeting inclusion criteria were fewer than anticipated. Despite their shortcomings, there is much to be learned in this area of inquiry. It is readily apparent that nursing programs planning to implement intervention strategies should require participation and offer them to all students. This approach would alleviate any detrimental effect of singling out a group of students and capitalize on the positive effects of additional faculty attention. In fact, it could be argued that faculty should treat all students as at-risk because nonacademic factors can jeopardize first-time NCLEX success for even the most academically gifted student. Exploration of why students are reluctant to participate in interventions and what factors might stimulate interest in and continuation with interventions would be helpful in designing future strategies.8
More research is also needed in minority and other understudied groups who are aggressively recruited into nursing yet may be particularly vulnerable to difficulties in passing the NCLEX examination17 and which may be magnified with the implementation of the new question format that requires typing in short answers.18
Software programs are increasingly available for purchase by universities. The use of NCLEX simulations and other computer-based testing indicate the ongoing desire of nursing programs to enhance student comfort with the structure and format of the licensing examination. In addition to the costs, instituting computerized testing programs can be time-consuming and frustrating in the absence of adequate technological support and resources.9 Well-designed research evaluating effectiveness of such strategies in a broad sampling of students is needed for a thorough cost-benefit analysis.
As NCLEX-RN pass rates continue to be viewed as an important measure of a nursing program's worth,13 research to identify interventions to promote success is paramount. Interest in predicting NCLEX success should continue, but equal effort must be directed to testing specific interventions used to help students prepare for NCLEX.
References