An end-of-semester course -reporting strategy was developed by our academic team to evaluate a recently implemented concept-based curriculum (CBC). The approach serves as one component of an overall curriculum evaluation plan and supplements other measures such as survey data, standardized testing scores, and NCLEX(R)-RN pass rates. According to Lee and Willson (2018), a comprehensive evaluation plan is necessary to ensure CBC quality. Because the move to a CBC required changes beyond content to include instructional techniques, the report-out strategy has been beneficial for evaluating our progress in this area. Report-out has also proven effective for sharing course-level data and outcomes related to course and program improvements. This article describes the report-out process and how it is implemented in a prelicensure baccalaureate (BSN) program. Each element of the process is described, and recognized gains are discussed.
The end-of-semester report-out strategy is modeled on clinical shift reporting methods. The current semester course faculty report to faculty who will teach students during the following semester. Because students complete the program as cohorts following a fixed plan of study, report-out is sequenced according to the curriculum's five-semester layout; Semester 1 reports to Semester 2, and so on. Administrators and other key personnel (e.g., simulation coordinator, BSN coordinator) attend each reporting session. All faculty attend report-out from the final semester courses because the information provides a comprehensive review of program outcomes.
Report-out is scheduled within a few days of the semester's end. The program's appointed BSN coordinator creates and distributes a report-out schedule to faculty. The report-out typically takes five hours with scheduled breaks. Each semester is allotted 45 minutes for their report, which is further divided among each course in the semester. On average, clinical courses are asked to complete their report in 15 minutes and nonclinical courses in 10 minutes. Faculty are enthusiastic about sharing their course reports and often exceed their allotted time. To stay on schedule, timekeepers are empowered to give a five-minute signal to end the presentation so receiving semester faculty have sufficient time to ask questions of the presenting semester faculty.
A report-out template is distributed to faculty to guide report organization. The template is broken down into teaching, testing, and student data. Report elements include integration of concepts in class and clinical based upon our program's concept-based curricular teaching framework, descriptions of active classroom teaching/learning strategies, testing data on concept performance, simulation performance data, course grades, standardized exam outcomes, and student feedback. The principal elements for report-out remain consistent, but additions can be made when needed. For example, during the pandemic, course faculty were asked to report on prepandemic and during pandemic instructional strategies. A written version of the report and accompanying support documents are compiled in a central course folder each semester that can be used for longitudinal or clustered assessments.
TEACHING
The first area course faculty address is the teaching framework used in class. This is divided into preclass, during class, and postclass work. Course faculty discuss the structured learning experiences provided and whether these experiences are synchronous or asynchronous. According to Scheckel (2020), well-designed learning experiences can promote students' critical reasoning skills and scholarly inquiry. Class learning experiences should be congruent with program, course, and module outcomes and competencies. If course faculty want to make changes for the next semester, they then address an action plan.
The second area is active teaching strategies to support the course concepts and exemplars. In the CBC, each course is responsible for presenting certain concepts (e.g., oxygenation, fluid and electrolytes, acid-base balance, culture, spirituality), along with specific exemplars that reflect the concept. Strategies that may be used to convey the concept and exemplars include unfolding case studies; NCLEX-style questions; compare/contrast; and games such as Kahoot, Jeopardy, and escape rooms.
The third area, clinical, encompasses laboratory, simulation, and clinical experiences in partnering agencies including hospitals, long-term care and rehabilitation facilities, and community organizations. The number of hours per student is reported in terms of clinical instruction; clinical laboratories for skill development; and simulation using high-fidelity simulators, telehealth robots, and virtual platforms.
Simulation outcome measures were developed by faculty in 2019; these measures became part of the report-out in 2020 to track student performance in each course with high-fidelity simulations. For example, in the fundamentals of nursing course, students performed above 95 percent in communication framework use, environmental safety, and the eight rights of medication administration. However, they scored below 75 percent on gas exchange (correct flow/assessment), assessment prior to medication, needle safety, and religious/cultural needs during death/dying/postmortem. This information provided stimulus for course changes: stress needle safety more in the laboratory, provide more hands-on training with oxygen delivery devices, and adjust scenarios to provide more application and judgment. As this cohort moves into the next course with 180 clinical hours, faculty can develop plans to fill in the gaps on these three competencies.
TESTING
Course grades and standardized test score distribution and mean are the last two areas under teaching and testing. The final distribution of course grades is reported. Exam analysis is conducted using ExamSoft (2019), which includes the exam mean, item difficulty index (p value), upper difficulty and lower difficulty index, discrimination index, point biserial correlation coefficient, and the Kuder-Richardson (KR-20) scores for each exam administered. This area has challenged faculty to look at their exams critically and to improve item-writing skills. The data have been beneficial for assessing program improvement initiatives focused on exam item writing.
It is important to remember that the purpose of a nursing exam is to assess proficiency by ensuring alignment between the students' expected knowledge, abilities, and skills and the test item. Faculty must test "need-to-know" content, write items with an appropriate level of difficulty, and administer a fair exam (Kranz et al., 2019). Two types of longitudinal reports of student performance on exams yield powerful evaluation information: 1) cohort performance in one semester of courses or across the program and 2) performance of items of individual courses over time. The most common use of the reports is the report-out of student performance on concepts in the current semester to faculty in the subsequent semester. A report of student performance on concepts across all semesters is used by faculty in the final semester to prepare students for the NCLEX-RN. A snapshot of program performance can be done by creating a report across all concepts in all courses at the end of each semester. For example, the COVID-19 pandemic caused hospitals to reduce clinical time or completely restrict students from clinical experiences. A longitudinal analysis of student performance on 59 concepts for the past three years showed three concepts that fell between fall 2018 and fall 2020 as a result of reduced clinical experiences (clinical judgment, 2018, 81.9 percent/2020, 78.3 percent; collaboration, 2018, 81.3 percent/2020, 75.6 percent; communication, 2018, 81.1 percent/2020, 77.5 percent).
The report-out strategy provides data for faculty in a subsequent semester to create a plan in the classroom, laboratory, or clinical settings to reinforce concepts with lower-than-expected performance in the previous semester. The template provides space for an action plan for the next semester. This approach empowers faculty to make course revisions that are based on evidence, and program performance has progressively improved. NCLEX pass rates, 84 percent in 2016, were 87.9 percent in 2018 and 95.3 percent in 2020. Standardized exam mean scores (>900 recommended) were 872 in 2016 and 918 in 2018; they fell to 908 in 2019 and increased to 921 in 2020.
STUDENT DATA
Student cohort feedback and student evaluations are examined to understand the student perspective. Cohort feedback is composed of responses course faculty receive from students throughout the semester; formal student evaluations are end-of-course evaluations addressing the didactic and clinical components of the course and course faculty. Course means are compared to college and university means.
CONCLUSION
Positive gains have been associated with the report-out strategy. The approach has helped us identify curricular strengths and weaknesses, encouraged instructional collaboration among faculty, and contributed to a successful curriculum transformation. The data have been used to inform decision-making and to enhance the overall curriculum evaluation plan. The strategy has also helped us maintain a steady course throughout the curriculum transformation process. Report-out could easily be adapted for various settings and is suitable at any stage of program development.
REFERENCES