The process of soliciting feedback from students about their teachers and the courses they take occurs every semester in, most likely, every nursing program in the country. Paper-and-pencil or electronic forms are distributed, reminders are sent to enhance response rates, data are analyzed, reports are prepared, and faculty take deep breaths before reading what students have said about them and the courses they teach. In many institutions, student evaluations weigh heavily in reappointment, promotion, and tenure decisions, and input from students can also prompt changes in the curriculum, including course focus, placement, or credits.
Student evaluations of courses and teachers are "easy to produce, administer, and tabulate" (Berrett, 2014, p. 1), perhaps fulfilling our need to quantify so many things in academy (Smyth, 2017). However, despite the ubiquity of evaluations and their potential impact on a faculty member's future, educators rarely engage in critical analyses of what is evaluated, the instruments used, the timing of evaluations, and how the findings are used.
Many issues persistently surround the practices of evaluation; "they remain controversial[horizontal ellipsis]and offer a false kind of security" (Berrett, 2014, p. 1). What is being evaluated? Are students qualified to make the judgments they are asked to make? Which students actually complete evaluations - strong students or those who are weak and receive poor grades? Other issues concern when evaluations are distributed, the format in which they are distributed, how much time students are given to complete evaluations, and how many evaluations they are asked to complete at any one time. Because of the high-stakes nature of student evaluations, interest in finding solutions to these issues is high, resulting in extensive literature, ranging from anecdotal reports of personal experiences to carefully designed studies intended to identify best practices.
In 2013, with grant support from the National League for Nursing, a group of faculty, students, and library staff colleagues at the Duke University School of Nursing initiated a systematic review to compile existing evidence regarding student evaluations of courses and instructors, particularly in relation to: a) issues surrounding evaluations, b) research related to course and instructor evaluation practices, and c) recommendations for best practices. The ultimate purpose of the project was to offer guidelines that could increase response rates of student evaluations while helping to ensure that such evaluations provide thoughtful feedback that serves to enhance the pedagogical expertise of instructors and strengthens course design and implementation to meet both student learning needs and program outcomes.
The systematic review of literature in nursing education, health professions education, and higher education (using ERIC, MEDLINE, CINAHL, and PsycINFO databases), as well as gray literature (e.g., Carnegie Foundation publications) and specific journals (e.g., Assessment and Evaluation in Higher Education), yielded more than 2,500 potential resources. After refining search terms, 1,589 abstracts were identified for review, and after applying the inclusion/exclusion criteria, 191 were identified for full-text review. The 191 articles fell into three primary categories: factors that influenced the completion or nature of evaluations, the content of evaluation forms, and how evaluations were used. Each was studied in greater depth by a subgroup of the study team.
As the subgroups analyzed and synthesized their findings, it became clear that despite reports of thousands of studies on the topic, little has changed regarding evaluation practices; in essence, issues raised long ago still remain. In fact, our systematic review revealed the same things Annan, Tratnack, Rubenstein, Metzler-Sawin, and Hulton (2013) reported from their integrative review of student evaluations of teaching:
* Students who expected a higher grade were more likely to provide higher ratings.
* Students who attended class provided higher ratings.
* Undergraduates are more critical of faculty than graduate students.
* Online classes received lower ratings than face-to-face ones.
* Students reported writing untrue things on their evaluations.
* Larger class size was associated with lower ratings.
* Courses with heavy workloads had lower ratings.
It was apparent from the findings that "the switch from paper to online [forms] has had a significant negative effect on response rates" (Annan et al., 2013, p. e20). Findings also revealed that the reliability and validity of instruments are questionable, and students may not interpret items or questions as faculty intended (Oermann, Conklin, Rushton, & Bush, 2018); many faculty do not believe that students take the evaluation process seriously; and faculty believe that, because students are not experts in pedagogy, the only thing they can reasonably evaluate is their own response to the course.
In light of such consistent findings - and limited change related to evaluation tools and processes - we believe it is time for the nursing education community to look seriously at our practices in this area. Questions such as the following need critical debate and must be studied through rigorous research:
* To what extent are the questions included in evaluation tools reflective of best practices in teaching? To what extent are they evidence-based? To what extent are they valid and reliable?
* Do all courses and teachers need to be evaluated through formal means each semester?
* Would formats such as open discussions with students - conducted in person, face-to-face, or virtually and facilitated by someone not directly involved in the course (e.g., a retired faculty member) - provide more meaningful feedback regarding students' perceptions of the course and/or a teacher's strengths and areas in need of improvement? Would such open formats provide opportunities to explore, in greater depth, student comments so their true meaning is evident and not open to interpretation by the readers of written comments? Would such discussions minimize false statements about what did or did not happen or what the teacher did or did not do?
* Would it be more beneficial to conduct more formal evaluations at the midpoint of a course so that feedback from students could be used to make revisions, adjustments, or improvements in the time remaining, thereby letting students see clearly how findings are used and why they are valuable? Such dialogue might address such elements as the clarity of course and teacher expectations, the purpose of the various assessment methods used, the clarity of evaluation criteria, and the relevance of the class or online topics addressed.
* Would faculty be able to better demonstrate their successes and innovations as teachers by providing reappointment, promotion, and tenure reviewers with narratives about how they planned and implemented the course, the innovations they used, and their degree of satisfaction with the outcomes, rather than merely submitting a number (e.g., 3.48 on a 4.0 scale) that summarizes the "bubbles" students have selected on an evaluation tool? Perhaps reviewers could be provided with an analysis that triangulates student evaluations, self-reflection, and peer review feedback.
In his book The Toxic University, Smyth (2017) bemoans the fact that faculty worldwide, in far too many institutions of higher education, fail to question the validity and value of many commonplace practices. Smyth primarily addresses the "corporatization" of higher education, the rampant use of metrics for things (e.g., learning) that are hard to quantify or things (e.g., scholarly impact) that cannot be fully measured merely by counting publications, and the extent to which the drive to achieve high rankings determines institutional policies, practices, and priorities more so than do the primary purposes of higher education. His criticism that faculty are remiss in not questioning or challenging "typical" practices can be easily applied to practices related to student evaluations.
We assert that it is time for faculty in schools of nursing to wake up and shake up. We must engage in serious discussions about why, when, and how we ask students to evaluate our courses and our effectiveness as teachers. We must engage in the struggle to define what "excellence in education" means in our particular setting. We must revise course and teacher evaluation forms based on evidence regarding good teaching practices and test them for validity and reliability. We must look to alternative or additional methods of evaluating teachers and courses, including critical self-reflection and peer evaluations. And we must look at how heavily we rely on student evaluations in promotion and tenure decisions that affect faculty advancement.
Only when faculty in schools of nursing do these things - and more - can we hope to attain robust, valid, and meaningful data that can be used to design curricula and create learning environments that exemplify excellence and evidence-based best practices. Without such rigor, systematic or integrative literature reviews will continue to document the same issues that just never seem to go away. We hope you will join in this effort to make the time-consuming and sometimes heart-wrenching processes related to student evaluations of courses and teachers more worthwhile and meaningful.
REFERENCES