Happy New Year! As the spring semester begins, faculty and students are beginning to focus on testing. Preadmission deadlines and end-of-program competency testing are lurking around the corner. January is also the time of year when programs start thinking about their testing needs for the fall and planning the next year's budget for those needs. Yes, test is one of those four-letter words that creates a flurry of activity and invokes many different thoughts and feelings, not only for the test taker but for the test administrator as well.
At National League for Nursing (NLN) Testing Services, we understand the importance of test preparation, test administration, and test results for both roles. Keeping in mind that tests are imperfect, there are practices and standards that should be implemented in nursing programs that administer exams, both criterion and norm-referenced exams. Since 1966, the American Educational Research Association, the American Psychological Association, and the National Council on Measurement in Education have published Standards for Educational and Psychological Testing, which is considered the gold standard for test developers, publishers, and administrators. (The 2014 edition is available at https://www.apa.org/science/programs/testing/standards.) The NLN, as a test developer and publisher, follows these recommended standards, and to keep current in the practices and recommendations for educational testing, our testing director is a member of these organizations.
When representatives of NLN Testing Services meet with nursing programs and health care institutions, the initial visit is one-on-one with a dean or program director. This visit is typically followed by a second meeting with a committee or group of faculty members (the testing committee) that are evaluating testing products for their program. Establishing the testing committee is where the standards begin. Do the members of the group or committee have education and training in assessment and measurement? There is little to no research on the criteria used by higher education for member selection to a testing committee. Most likely, departments set internal policies for membership or ask for volunteers. Test committee selection is an area to consider for future research.
Before meeting with vendors, the testing committee may formalize a questionnaire. Although the hope is to compare apples to apples, vendors' products, like apples, come in many varieties. The survey will ask types of products offered, perhaps publications and research, remediation opportunities, and, of course, cost. All are fair questions, but does the vendor questionnaire provide the testing committee with the facts and data that will help them determine if the vendor's products will meet the testing needs of the program? When meeting with a test vendor, the testing committee should have a clear understanding of the following: the purpose of the exam, testing policies, and testing procedures.
THE PURPOSE OF THE EXAM
Most exams are multipurpose and may be criterion-referenced (CRT) or norm-referenced (NRT). CRT exams are administered to determine the student's success in performing or learning a set of competencies, objectives, or tasks. A CRT score is commonly the number of items correct/total number of items, generally shown as a percentage correct (maximum score = 100 percent). Norm-referenced exams compare the student's success for performing or learning a set of competencies, objectives, or tasks against peers also undergoing the examination.
A norm-referenced score is reported as a ranking or percentile rank. Percentile ranks indicate the percentage of students in the norm (reference group) who scored the same or lower than the achieved score. A student at the 50th percentile reached the same level the norm group averaged.
Exams may also be used to evaluate curricula or instruction at the institution, campus, or regional level. An exam may be used to make inferences, and caution should be used to ensure the validity of these inferences. The purpose of one exam may not be the same as another, and so the goal for each exam type should be considered.
* What are the stakes? Is the exam a high-stakes exam in which the individual's score will have a significant impact? Or is it a low-stakes exam where the scores are informational but do not affect the test takers' future paths?
* Is the exam mandated by the government, institution, or department, and what are possible intended and unintended results?
* Does the functionality of the test administration system and publisher policy and procedures coincide with department and institution testing policy?
TESTING POLICIES
While meeting with the testing committee, NLN Testing Services will ask about the program's testing policies. Typically, there are policies for absence and make-ups and unethical behavior, but having a current, comprehensive written plan that is communicated and acknowledged by faculty, staff, and students may or may not exist. NLN Testing Services receives an average of 10 or more communications per day from institutions and students requesting information about test registration, scoring, retakes, and interpreting score reports - the type of information that should be included in a program's testing policy. NLN Testing Services offers an on-site workshop for establishing testing policies and also during the NLN Education Summit.
TESTING PRACTICES
Identification of a testing vendor or vendors should not be the only responsibility of a program's testing committees. Members should review and consider the other recommendations under Part III, Standard 13 in the 2014 Standards for Educational and Psychological Testing. These practices include the following:
* Establish local norms. When onboarding a new customer, NLN Testing Services recommends that a group of established students (high, low, and average performers) administer the exam(s). This practice is especially helpful when setting initial scoring expectations.
* Review high-stakes exams to ensure there is evidence that students had learning opportunities to cover both content and skills. High-stakes exam publishers should provide proof of both national content and skill review, as well as bias review. The NLN has established two separate committees of national nurse educators to ensure the quality and generalization of the items in our test banks to address these elements.
* Establish policies for retaking exams. When required to demonstrate mastery, students should have an established number of retake opportunities that have been deemed reasonable. NLN Testing Services defers to the institution to set retake policies as the institution has the best understanding of the test taker population.
* Use exams in conjunction with other forms of assessment. A single exam score should not be the sole source for consideration when making a decision that significantly impacts the test taker. NLN Testing Services recommends exams be used in conjunction with other forms of summative and formative assessment.
* Score reports should be easy to read and interpret. The data reported should include the date administered, the age of the norms, and the standard error of measurement. The smaller the standard error of measurement value, the more accurate the assessment. Many institutions will simply record a test taker score, but a score report is more than a number. It is a learning tool, and those responsible for receiving score reports should have knowledge and training in analyzing and utilizing the data in that manner.
NLN Testing Services is committed to continuing research and conversations about assessment and evaluation in nursing education. We hope that faculty in nursing education and professionals in the assessment and measurement field will join NLN Testing Services in identifying current standards and establishing new standards and practices for testing as a learning tool in nursing education. There is a ripe opportunity to develop a community focus on assessment and evaluation, and NLN Testing Services would like to partner with your program in achieving the gold standard.