Maternal mortality rates in the United States have been steadily rising over the past 2 decades and are substantially higher than those in most other developed nations.1 There are also significant disparities in maternity outcomes, with Black and older gravidas having significantly higher mortality rates.1 These dire statistics have prompted calls for improving care practices throughout the perinatal period to reduce maternal mortality, morbidity, and disparities in perinatal outcomes. Quality improvement (QI) initiatives such as peer review of clinical practice are essential to address this perinatal crisis.2 Reviewing peers' work contributes to effective care from nurses, advanced practice nurses, physicians, and other perinatal care providers.2
Peer review is a hallmark of professional practice, and professions that do not self-regulate by using peer review are subject to external regulation.3 The American Nurses Association endorses peer review and has established guidelines for using peer review as a part of QI initiatives.4 When effective self-regulation using peer review is employed, patient harm is reduced.5 Self-regulation requires education on peer-review processes with a focus on personal accountability and system evaluation to increase patient safety.6
In addition to peer review, effective QI initiatives include the development of a just culture.7 As part of recent safety initiatives, a just culture movement has emerged in 79% of acute care hospitals in the United States.8 In organizations that use a just culture, individuals are expected to report errors and demonstrate personal accountability for improving care, and the organization assumes the responsibility for improving system processes.2,5-13 Employees are expected to monitor themselves and each other to ensure safety for all.10,13 Perinatal care providers and students must be educated about the need for and how to comply with guidelines using a just culture to ensure safety in perinatal care and healthcare for future generations.14,15
Simulation is a valuable learning strategy that provides the opportunity to practice clinical and communication skills in a safe environment and to receive constructive feedback before performing the skills in clinical practice.10,16-18 Although the benefits of simulation learning experiences (SLEs) are well documented, students in online and distance education programs often have fewer opportunities to participate in SLEs. Conducting simulations from a distance allows for more accessibility for learners while not detracting from their learning so that they can have meaningful additions to their education.19 However, educators must be aware of technology needs that can add to learners' cognitive load, especially in simulations delivered from a distance. Cognitive load is increased by the extraneous knowledge and skills the learner needs to participate in the simulation but are not the activity's primary focus. Keeping cognitive load to a minimum allows students to focus on the intended learning that needs to happen in the situation.20
Although the benefits of SLEs are well documented in the health sciences, a literature search of MEDLINE, CINAHL, APA, and ERIC, using the terms "peer review," "just culture," "patient safety," "chart review," "quality improvement," and "quality assessment," identified little research regarding teaching peer review. The lack of research on teaching nursing peer review is problematic, and nursing lags behind other health professions in implementing robust programs clinically.21 Thus, there is a need to teach the processes of conducting peer review of health records using just culture principles and communicating the findings of the reviews in a safe environment. The purpose of this project was to evaluate students' experiences participating in a peer-review SLE using just culture principles in a graduate-level, online distance nursing education program.
METHODS
Design
This mixed-methods study was originally designed to evaluate an educational intervention to provide a peer-review SLE for graduate-level nursing students in advanced practice nursing and doctor of nursing practice (DNP) programs. The peer-review SLE was pilot-tested and evaluated in the fall of 2017 and became a required course learning activity in subsequent terms.
Participants
A total of 4175 students completed the peer-review SLE over 19 terms in a 5-year period. The students were all registered nurses with professional nursing experience and were enrolled in a master of science in nursing (MSN) advanced practice nursing educational program (nurse-midwifery, women's health nurse practitioner, family nurse practitioner, or psychiatric mental health nurse practitioner program) or a DNP program designed for advanced practice registered nurses (APRNs). All students were required to participate in the peer-review SLE as part of the course requirements.
Of the 4175 students participating in the peer-review SLE, 3823 (91.6%) completed the SLEI survey. Most surveys were fully completed, with minimal missing data. The percentage of missing data for the 32 items on the SLEI was low overall, varying from 0.1% to 0.6% missing data per item.
Instrument
Students' perception of the peer-review SLE was assessed using the Simulation Learning Effectiveness Inventory (SLEI), a validated instrument designed to measure learning effectiveness and students' perception of the SLE.22 The SLEI was modified for this assignment with permission from the instrument's author. The modifications included changing the words "course" to "assignment," "equipment" to "technology," and "situational learning" to "simulation." The SLEI contains 32 items that measure learning effectiveness and the learning experience in 7 domains: assignment structure, technology resources, debriefing, clinical ability, confidence, problem-solving, and collaboration. Students rated their experience and confidence on a 5-point Likert scale, with 1 representing "strongly disagree" and 5 representing "strongly agree." One open-ended question was added to the SLEI that asked students to provide any additional feedback about the experience of participating in the simulation. Three multiple-choice questions were also included to obtain information about which program students were enrolled in (MSN or DNP), whether they had previously participated in peer review (Yes or No), and their role in the SLE (peer reviewer, QI committee chair [QI chair], or observer).
Procedures
Each peer-review simulation involved 8 students who were grouped into 4 pairs. The simulations focused on one of 4 aspects of healthcare in the perinatal period: breast cancer screening, depression screening, intimate partner violence screening, or acute rhinosinusitis treatment. All 8 students in an SLE addressed the same topic. Before the simulation sessions, students were asked to identify and review an evidence-based guideline for their assigned clinical topic. All students were given simulated health records to review to determine whether the healthcare providers had met quality standards related to the assigned clinical topic.
Students were prepared for their roles in the SLE before the scheduled session. One student was assigned the role of the peer reviewer with responsibility for examining the simulated records for the department, and the other student was assigned the role of the chairperson of the institution's QI committee (QI chair). In the SLE, the peer reviewer was responsible for presenting the results of their review to the QI chair, creating a power differential between the peer reviewer and the QI chair. Before the SLE, students participated in a briefing session with a faculty member who provided instructions and guidance for the role they would be playing in the SLE. Students in the role of the QI chair were provided with video and written instructions about points to make during the experience to ensure that the SLE unfolded in a predetermined manner. Peer reviewers were required to address common questions encountered when evaluating peer-review results. Unknown to the students who served as peer reviewers, students playing the role of the QI chair were instructed to insert pushback comments during the conversation to facilitate the peer reviewer's critical thinking. If a student missed a scheduled session, the remaining student from that pair became an observer for one of the other 3 simulation pairs and was given specific instructions about what to observe during the simulation. Students in the observer role could then participate fully with the other students in the debriefing session.
All SLEs and debriefing sessions occurred online in an interactive videoconferencing environment. Prebriefing was performed for all 8 students in the main virtual room and then student pairs were placed in individual breakout rooms and given 15 minutes to complete their simulated peer-review encounters. All students and faculty members then returned to the main virtual room for a 30-minute debriefing session. Although students were required to participate in the experience, they were not graded on their performance during the simulation encounters since the instructional focus was on providing the opportunity to practice peer review, conflict management, and just culture in a formative, low-pressure setting.
The debriefing session addressed the students' general impressions, principles of just culture, navigation of difficult conversations, and power differentials in the clinical environment. The students offered each other feedback on interpersonal communication skills, professionalism, and the use of just culture during the simulation. Emphasis was placed on the many acceptable approaches to handling difficult conversations with colleagues and the value of peer review and just culture.
After the debriefing session, students completed the SLEI anonymously using the SurveyMonkey platform. Although students were not required to complete the questionnaire, those who did so received 5 points out of 300 total course points (1.67% of the total course grade), which provided a small incentive for survey completion. Student evaluations of the peer-review learning experience were not reviewed by faculty until after each term was completed, grades were submitted, and students were no longer enrolled in the course.
Qualitative analysis
Qualitative data were mined for themes using Excel version 2010. The first round of coding was performed by 2 of the authors (L.C. and E.T.), and descriptive, in vivo, and process codes were identified utilizing Saldana's method of first-cycle coding.23 The coders met and identified similar themes and a common language for concepts based on overlap between the codes. The second cycle of pattern coding was performed by both authors, using the codes identified in the first cycle. Pattern codes were used to develop similar themes and used as cross-references to support and elaborate on quantitative data outcomes.
Ethical considerations
This project was started as an educational intervention and the SLEI was used to evaluate whether the peer-review SLE was effective and acceptable to the students. Data from the questionnaires were retained as part of the students' course performance records. Over time, student responses to the experience indicated that the SLE was an effective and positive learning experience and the course faculty decided that this information should be shared with other educators. The project was then submitted to the Frontier Nursing University Institutional Review Board for review and was determined to be exempt from review since the data could be retrieved with no student or other identifiers.
Statistical analysis
Data were analyzed using SPSS version 29. Descriptive statistics were performed for all variables and the 32 items and 7 domains of the SLEI. The Mann-Whitney test was used for comparing the experiences of students in the MSN and DNP programs and for comparing students with and without prior SLEs. The Kruskal-Wallis test was used to compare the experiences of students who assumed the roles of peer reviewer, QI chair, and observer in the learning experience. A posteriori comparisons were performed on the statistically significant findings, and [alpha] was adjusted using Bonferroni's correction for multiple comparisons.
RESULTS
Quantitative results
The SLEI was completed by 3823 students, which included 3322 (86.9%) MSN students, 497 (13%) DNP students, and 4 (0.1%) students who did not identify their educational program. Previous simulation experience was reported by 1625 (42.5%) students, no previous experience was reported by 2193 (57.4%) students, and 5 (0.1%) students did not respond to the question. The students reported their role in the learning experience as serving as the peer reviewer (n = 1828; 47.8%), QI chair (n = 1837; 48.1%), or observer (n = 150; 3.9%), with 8 students (0.2%) not reporting their role.
The SLEI scores for the 7 domains of the inventory are presented in Table 1. The scores are high in all domains, with the mean score approaching the highest possible score for all items in the domain and the mode being identical to the highest possible number of points in each domain.
The SLEI scores for MSN and DNP students are compared in Table 2. The MSN and DNP students had very similar mean scores in all 7 domains. Although the DNP students had slightly higher mean scores than the MSN students in 5 domains, the only statistically significant difference was in the domain assignment structure, with DNP students having significantly higher scores.
The SLEI scores for students with previous SLE and those without previous SLE are compared in Table 3. The 2 groups of students demonstrated similar mean scores in all 7 domains. Although the students with previous SLE had slightly higher scores in 5 domains, the only statistically significant difference was in the domain assignment structure, with students who had previous SLE having significantly higher scores.
The SLEI scores of students who played the roles of peer reviewer, QI chair, and observer are compared in Table 4. The scores of the 3 groups of students were similar, with those in the peer reviewer role having slightly lower scores than those in the QI chair role in 6 domains. The observers had the highest scores of the 3 roles. Although the differences between the groups were very small, the students in the role of peer reviewer had significantly lower scores in 3 domains: assignment structure, clinical ability, and confidence.
Qualitative analysis results
Qualitative analysis identified 3 major themes: meaningful learning experience, real-world application, and technology challenges. Subthemes supporting each theme were also identified. The identified themes and subthemes supported the findings of the quantitative data.
Meaningful learning experience
The value of the peer-review simulation as a learning experience was frequently expressed in the qualitative data. Students commented on how enjoyable the simulation experience was despite having felt anxious about it beforehand. One student commented, "This was an interesting exercise that I expected to be unhelpful due to its scripted nature. The individual I was paired with took it seriously and we had an excellent discussion and it felt natural." The most used phrase to describe the value of the simulation was "eye-opening." Comments such as "This was an eye-opening experience into the process of peer review and how it can promote a just culture" appeared repeatedly. The debriefing portion of the simulation was significant for participants: "I really learned a lot from this experience. I especially appreciated the debriefing session following the peer review. Group reflection is so helpful for self-understanding and growth."
Real-world application
The identified theme of real-world application further supported the value of the peer-review simulation experience. One student's comment was typical of this theme: "It was helpful to learn what types of questions and concerns could arise during the peer review discussion. It was beneficial to practice how to approach and manage those topics for real-life practice." The value of teamwork was integral to the real-world application of the simulation:
This was a really great experience. I learned principles that I will use when I practice. I have gained an under-standing of the importance of following guidelines and utilizing communication to work with others in providing excellent, safe patient care. This occurs through teamwork and not one person. Thank you!
Technology challenges
The final theme identified was related to technological difficulties students experienced before and during the simulation experience. The simulation assignment consisted of multiple components presented in an online setting. Students found the multiple steps in the process to be confusing, and this was confounded by the virtual environment. The most common challenge related to the simulation was connectivity during the synchronous event. One student summarized this saying, "Seemed to be significant problems with connectivity during the session. I know some of it was related to my Internet connection, however, others were also experiencing problems with hearing the instructor clearly." However, the simulation remained a meaningful learning experience: "These simulations are effective learning forums. Ours was interrupted by technical difficulties but was still effective."
DISCUSSION
Peer review and just culture are important components of patient safety and QI initiatives, and students should learn the process of peer review and the principles of just culture regardless of whether they are entering or advancing in the nursing profession. The findings of this study suggest that using simulation learning to teach peer review is effective in a graduate-level, online distance nursing education program. Providing an opportunity for students to practice peer review and communicate their findings and recommendations for systems-level improvement in a simulated environment contributes to student understanding of important lessons for future professional nursing practice.
The SLE provided students with a safe place to practice their communication techniques when a power differential was evident. The debriefing session helped students process the experience and find relevance to future practice. Other studies have reported that meaningful learning occurs in video-based online simulation activities.19,24 The very high ratings on the SLEI and the positive student comments in this study provide evidence that the learning activity was effective and meaningful for both MSN and DNP students.
Using an online SLE in a distance education environment created some technological challenges for students and faculty. Internet connection issues were reported by some students but were relatively infrequent overall. Technology challenges such as connectivity issues may have increased cognitive load for students who experienced these problems. Indeed, a recent integrative review concluded that task complexity and distractions can lead to increased cognitive load.20 When the technology issues were identified, the issues were addressed by having an employee of the information technology department of the university attend each simulation session to help students resolve issues. Instructional design experts within the university also provided guidance to the faculty for navigating these challenges. Student feedback on the SLEI each term identified connectivity as a problem, so changes were made to streamline the assignment steps and to continually improve the learning experience.
The SLEI focuses on evaluating the learning experience, its ability to meet the learning objectives, and preparing the learner for real-world application in future practice. Some of the questions asked the learner to determine whether the simulation contributed to the mastering the peer-review process, application of knowledge about the topic, practice for future real-life encounters, and problem-solving techniques for future encounters. The findings that doctoral students and students with prior peer-review experience had significantly higher scores in the domain assignment structure suggest that prior exposure to simulation learning and being an experienced APRN may increase students' appreciation of simulation learning and the importance of peer review.
While all 3 groups of learners scored highly in all 7 domains of the SLEI, the students who played the role of peer reviewer had significantly lower scores in 3 domains: assignment structure, clinical ability, and confidence. These findings suggest that the role of peer reviewer may be more stressful and these students may require more support throughout the SLE.
This study's findings are consistent with the literature on the value of simulation in nursing education.17,18 Students found that the activity broadened their thought process to ideas they did not hold prior to the simulation experience and promoted problem-solving skills that can be used in patient safety situations in future practice. The student scores on the SLEI and their written comments indicate that the SLE met the desired goals.
This study was limited by data collection in one graduate-level nursing education program. Another limitation is that more detailed demographic information such as years of professional nursing experience was not collected. Although this simulation was developed for one graduate-level, online nursing education program, the findings of this study suggest that using simulation learning to teach peer review may be effective at all levels of nursing education, including perinatal and neonatal nursing students and nurses in the clinical setting.
There is still much work to be done to increase patient safety in healthcare. One key element in the success of safety initiatives is changing peer review from a punitive event to one that examines the entire system with an emphasis on patient safety and just culture.5 Other important factors include professional self-regulation by peer review.3 Although this study was conducted with graduate-level nursing students as participants, concepts simulated here are applicable to nursing learners of all levels, as well as nurses in clinical settings. Having the ability to learn about and practice skills in a safe environment is helpful to facilitate future patient safety initiatives. There are many approaches to facilitating the peer-review process, and more research is needed to determine best practices.
References