Interrater concordance in the assessment of student writing is a desirable attribute within an academic program. It requires that faculty share common standards of excellence and measurement of student performance. However, assessment of student writing is historically viewed as a largely subjective task (Kohn, 2006; Wilson, 2007) wherein faculty must necessarily exercise independent judgment resulting in a process that Kohn deems both "imprecise" and "subjective." Therefore, differences in grading standards and outcomes are commonplace. For students, this means that achieving excellence requires the pursuit of a moving target with inconsistent or unreliable standards.
Although rubrics are commonly recommended to enhance grading consistency in the assessment of a single assignment (Nilson, 2016, Schinske & Tanner, 2014), standardized versions with uniform expectations are seldom used across a curriculum. In fact, such standardization has the potential to conflict with faculty academic freedom to design coursework. Collaboration between humanities and nursing faculty in an RN to BSN program has led to the development of a standardized writing rubric model that is appropriate for use in writing assignments across the curriculum and provides palatable levels of academic freedom to faculty.
BACKGROUND
Nurse educators are charged with the responsibility of supporting students in the development of proficient and professional writing skills. Even so, educators can be reluctant to assign writing projects because of the time-consuming nature of grading such assignments and the inherent subjectivity present in the grading effort. To encourage the development of student writing skills, many institutions have implemented Writing Across the Curriculum programs that require faculty to issue writing assignments in all courses (Porter, 2016). In the assessment of writing assignments, great variance in standards is typical, leading some authors to distance themselves from the goal of consistent marking (Bloxham, den-Outer, Hudson, & Price, 2016).
Consistency and interrater concordance, however, are strongly associated with fairness in nursing education (National League for Nursing, 2012). Rubrics are considered best practice to promote consistency and interrater reliability in the assessment of a single assignment (Nilson, 2016). Typically, rubrics are designed by each course lead instructor, and they are tailored to meet the needs of each writing assignment topic. Grading rubrics change from assignment to assignment and from course to course.
This tradition supports significant variance and inconsistency in grading standards, and consistency in standards is a necessary condition for consistency in grading itself. However, imposing an institutional rubric upon faculty restricts their ability to set appropriate standards for course activities. As such, institutional rubrics may present an implied threat to academic freedom, a freedom that protects a faculty member's rights to determine the appropriate grade for a student (Nelson, 2010).
DESIGNING OUR RUBRIC
Faculty members in our RN to BSN program come from diverse educational specialties that include humanities, the sciences, and nursing. In faculty meetings prior to the development of a standardized rubric for writing assignments, instructors agreed that interrater concordance and common standards of excellence were important. There was a consensus that unified standards foster a more productive, healthy learning environment for students and faculty. To find common grounds in faculty rubrics, the assessment tools used in assigned writing coursework were collected as voluntarily submitted by faculty.
We found no significant common ground in the rubric standards. Some faculty issued rubrics in list format; others used tables. Desirable attributes in professional grammar and formatting were sometimes disclosed as a general category; other times, instructors sought particular skills, such as citation formatting. On occasion, formatting standards and grammar were not assessed at all. In terms of conceptual development, grading rubrics showed no similarities; such variances in expectations were anticipated, given the variance in subject matter.
To create a more concordant approach to grading papers, a humanities educator and a nursing educator developed and introduced a model for a standardized rubric system to be employed across the curriculum. The standardized grading rubric system allows for individualized, potentially subjective judgment in areas that are unique to each course. At the same time, the grading rubric also designates standardized sections for matters of objective judgment. For example, such standardized areas of evaluation include whether a title page is appropriately formatted according to the standards of the 2009 American Psychological Association (APA) 6th edition publication manual; adherence to the standards of an institutionally accepted publication manual are matters of fact, not opinion.
The proposed rubric system has the following characteristics:
* Seventy percent of the points for any assignment are customizable and always independently determined by the course instructor. The content row of the rubric hosts these points, and instructors are at liberty to edit the row to include expectations that may include adequacy of analysis, sources, or application of theories.
* The remaining 30 percent of the points are divided between grammar and specific requirements for formatting in accordance with the publication guidelines of the APA.
* Students are expected to learn new APA skills in each unit. The rubric system differentiates standards for emerging APA skills from standards for APA skills that have been mastered in prior units.
* The list of expectations within the standardized 30 percent expands as students advance through the coursework in the four-unit program. That is, rubrics in Unit 4 courses are more demanding than rubrics in Unit 1. (See Supplemental Digital Content 1 at http://links.lww.com/NEP/A81 for examples of rubrics for Units 1-4.)
APPLICATION
In permitting faculty to maintain control of 70 percent of the points on any paper, academic freedom is satisfactorily preserved in the rubric, as further documented by the unanimous acceptance of the proposed rubric in a faculty meeting. The instructor-determined 70 percent of assignment points may be issued based on conceptual success, use of evidence, argument formation, or any other appropriate categories for the assessment of a writing project.
The 30 percent that is standardized in the rubric system is designed to optimize grader consistency on objective issues. The APA publication manual has 272 pages; data suggest that even experienced faculty may lack encyclopedic knowledge of its rules (Onwuegbuzie, Combs, Slate, & Frels, 2010; Savor, 2017). The model rubric system emphasizes rules within the program that are commonly enforced. Although rules regarding the space between equal signs may be low priority, other attributes, such as the appearance of the title page, are typically important. Ultimately, the collaborative rubric identifies title page, running headers, and reference, citation, and subheading formatting as areas in which students should demonstrate competence by the completion of the program.
A happy byproduct of the standardized rubric system is a model with improved usability for students. Rather than negotiating a new set of standards and style of rubric for each writing assignment, the model presents students with information in a format that will remain consistent throughout their academic careers. In essence, the formatting of rubric expectations remains constant throughout a program, just as the formatting of a patient chart remains constant within a health care institution.
CONCLUSION
Both writing skills and consistency in evaluation are valued in nursing education. Although some facets of writing evaluation may always be subjective, this theoretical model promotes consistency in the evaluation of areas that are objective, such as conformity to formatting rules. It is both authors' hope that these models introduce the reading audience to a potential area of compromise between the objectivity of standardization and academic freedom while providing a solid foundation for the pursuit of interrater concordance.
REFERENCES