View Entire Collection
By Clinical Topic
Diabetes – Summer 2012
Future of Nursing Initiative
Heart Failure - Fall 2011
Influenza - Winter 2011
Nursing Ethics - Fall 2011
Trauma - Fall 2010
Traumatic Brain Injury - Fall 2010
Fluids & Electrolytes
This article presents the implementation of secure laptop-based testing in an undergraduate nursing program. Details on how to design, develop, implement, and secure tests are discussed. Laptop-based testing mode is also compared with the computer-laboratory-based testing model. Five elements of the laptop-based testing model are illustrated: (1) it simulates the national board examination, (2) security is achievable, (3) it is convenient for both instructors and students, (4) it provides students hands-on practice, (5) continuous technical support is the key.
Until the 1990s, paper-and-pencil tests had predominated in nursing education. Halkitis and Leahy1 emphasized the importance of nursing students practicing with computerized testing before taking the licensing examination for nursing. Bloom and Trice2 conducted a study comparing nursing students' performances on paper-and-pencil tests with those who took computerized tests. Their findings noted that both groups of students did equally well. Bugbee3 also demonstrated that paper-and-pencil- and computer-administered tests are equivalent, especially when the tests are the same. Computerized testing in the classroom became possible due to more and more sophisticated learning management systems.4 Computerized testing has also been found to be more efficient in that students need less time to take the test.5
Until recently, most nursing programs administered paper-and-pencil tests to students despite the fact that the NCLEX-RN in the United States has been computerized since 1994. According to Anna,6 testing on computers in nursing education was still considered an emerging application at the end of the 1990s and at the beginning of the 21st century.
Reising,7 however, pointed out in 2003 that there were no significant differences in NCLEX-RN passing rates between the students who were exposed to computer-based testing in their nursing programs and those who were not. The massive use of NCLEX-RN simulations and a growing number of computer-based testing programs by colleges across the United States, however, indicate the ongoing desire of nursing programs to enhance student comfort with the structure and format of the licensing examination.7 For example, Jacobs and Koehn8 reported using Assessment Technologies Institute service to implement computerized testing to provide nursing students experience with standardized computerized testing and to increase the students' NCLEX-RN passing rate. Vrabel9 identified several advantages of desktop-based computerized testing in 2004: scheduling convenience, instant scoring capability, and enhanced security. Vrabel9 also identified one possible disadvantage of an increase in testing anxiety for those without previous experience, although results in this area were mixed. Some other research has shown that anxiety based on computerized testing actually decreases with hands-on practice on the computer.10
One way to implement the computerized testing is to use the computer laboratory on campus. For example, in 2003, the College of Business Administration at the University of Central Florida (UCF) opened a computer-based testing laboratory to facilitate administration of course examinations. In 2008, more than 120,000 examinations were administered. The major benefit of utilizing a testing laboratory is its increased security, due to the proctor's presence and the fact that students leave the computers after the test. However, consistent support and scheduling can become quite challenging in the context of increased demand.11 In addition to the cost, instituting computerized testing laboratories can be very time-consuming and frustrating in the absence of adequate technological support and resources.12
Ever since Angel Secure Browser 2.0 (Angel has been acquired by Blackboard Inc., Washington, DC) was released in 2008 by Angel Learning (http://http://www.angellearning.com), secure testing on students' own laptops has become a viable option. Angel Secure Browser can be installed on a student's laptop, and when it is launched, the desktop screen is locked so that students cannot do anything except take the examination. There are two weaknesses of Angel Secure Browser: (1) it is Windows based and cannot work with Macintosh operating systems and (2) it is a feature offered only by Angel Learning Management System (ALMS). Respondus Lockdown Browser (Respondus, Redmond, WA), however, was released shortly after, and it became a viable option for schools that are using other learning management systems (LMSs) such as Blackboard-Learn (Blackboard, Washington, DC), Moodle (an open-source platform available at moodle.org), or Sakai (another open-source platform offered by the Sakai Foundation, sakaiproject.org), and it is compatible with both PC and Apple computers.
As more and more campuses migrated to the blended learning mode that utilizes a sophisticated LMS, along with the widespread increase of students' personal laptop use, laptop-based testing (LBT) has become a viable option in administering quizzes or tests. However, little research has been done specifically addressing the pros and cons of this LBT mode in nursing programs. This article presents a specific case of how an undergraduate nursing course from a college in the United States designed, developed, implemented, and evaluated a secure LBT program. Surveys were conducted after two secured examinations to collect students' feedback regarding the testing format, and data were analyzed. Five elements of the LBT program are discussed at the end of the article.
With computer-based testing in NCLEX as a driving force, the authors' college embarked on a laptop initiative circa 2004. The nursing instructors felt that timed, computer-based assessments would better prepare the undergraduate nursing students for what they would face during licensure examinations. The notebook/laptop was the chosen device for the students because it was portable. Laptops can be used not only for testing, but also in teaching and learning. The nursing instructors and the information technology (IT) department worked together to develop an image set for the custom-imaged Dell Latitude model (Dell Computer, Round Rock, TX). A wireless network was installed and configured for the college. As the campus migrated from traditional face-to-face teaching to blended-learning mode, various computer/Internet-based technologies were introduced to the classroom. The laptop initiative is the "one stone that killed two birds."
Secure testing is made possible by the following two conditions. (1) A robust LMS that can allow instructors to create and host all the test questions is required. Features such as randomization, one question at a time, specific testing time setting, automatic self-grading, and regrading are also necessary. (2) The test questions need to be administered in a secure browser where opening new windows, minimizing windows, right clicking, or printing will be prevented.
The authors' college is currently utilizing ALMS, now called Blackboard-Angel after the Blackboard acquisition of ALMS in 2009. All the test questions are designed, developed, and hosted within the ALMS. The ALMS offers a variety of question formats such as multiple choice, multiple select, ordering, matching, and fill-in-the-blanks. Most of the nursing tests are composed of multiple-choice questions; however, with NCLEX moving toward alternative formats such as multiple select, ordering, matching, and fill-in-the-blanks, more and more instructors have begun to include those alternative questions in their examinations. For example, dosage calculation questions in all nursing courses are currently using the fill-in-the-blank format, instead of the previous multiple-choice format. The instructor has full control of testing settings such as time frame of the test, randomizations, attempts, browser security level, all questions at a time or one question at a time, and review setting. Other LMSs in the US market such as Blackboard-Learn, Moodle, or Sakai can achieve similar testing features and functionalities according to EduTools LMS comparison spreadsheet.4
Angel Secure Browser, which is a feature inside the ALMS, was installed on every nursing student's laptop. The Angel Secure Browser disables right clicks, printing, and saving. It locks down students' laptop screens so they can only take the test and are not able to visit other Web sites for references during the test. If other LMSs such as Blackboard-Learn, Moodle, or Sakai are used to implement secure testing, Respondus Lockdown Browser can be installed on students' laptops to conduct secure testing. The Respondus Lockdown Browser is superior to Angel Secure Browser in two major ways: it can incorporate a simple calculator for the dosage calculation questions, and it is compatible with the Macintosh operating system.
Starting the fall 2004 term, the laptop initiative at the nursing department required every entering nursing student to purchase a college-deployed laptop. The laptop initiative paved the way toward secure LBT: first, it allowed the college to preinstall Angel Secure Browser on every student's laptop before the actual deployment; second, because of the same brand and model of the laptop deployed to the students, trouble-shooting students' laptop-related glitches became more manageable.
Tests are given in the classroom (Figure 1). Students bring their laptops to the specified classroom at a given time. In this case study, there were 60 students enrolled in the Foundations of Nursing class for the summer term in 2010. Two instructors were present to proctor the examination. Students were asked to clear their desktops leaving only the laptop, a simple calculator, a pencil, and a piece of scratch paper for dosage calculations. All students were asked to use the bathroom before the examination, because once the examination started, no bathroom breaks were allowed. Students then were asked to run the Angel Secure browser. They then logged in and saw the examination title. Once the instructors released the password to the class, students proceeded to work on the questions on the test.
Security is the number one concern when it comes to any testing in the nursing program. In this case study, measures to increase test security were summarized based on continuous feedback from students, instructors, and instructional technologists from the author's college. They are listed in Table 1.
For the Foundations of Nursing course, the instructors decided to display all questions on the students' laptop screens at one time so that students could go back and forth to change answers. Their intention was to reduce students' anxiety because it was the first nursing course in the nursing program to implement the LBT. The rest of the nursing courses adopted the NCLEX testing style: one question at a time on the screen and no backtrack. Additional advantages of using LBT are listed in Table 2.
A survey was implemented after students finished two secured unit examinations in the Foundations of Nursing course in the middle of the 2010 summer term. Interviews with instructors were also conducted so feedback on the management and security of the testing can be obtained. Both quantitative and qualitative data from students were collected through a piloted survey, which was located within the LMS. Fifty-six valid survey results were collected in this course with 60 students enrolled. The survey had 10 questions, nine questions of which were multiple-choice questions that used a Likert-scale rating of 1 to 5 (1 = strongly disagree, 5 = strongly agree). Among the 56 respondents, 51% of them were identified as white American; 24%, as Hispanic American; 20%, as African American; and 6%, as Asian American. There was one open-ended question at the end of the survey aimed at collecting students' detailed reflections and feedback from their LBT experience (Table 3).
Five elements of laptop-based testing emerged after detailed data analysis:
1. It simulates the national board examination. Most of the students were in favor of this LBT program, and they agreed that the computerized testing format would help them better prepare for the national board NCLEX-RN examination. Student A made the following comment in the survey:
I think the computer testing is a great idea! It will provide the groundwork for the NCLEX-RN exam and it will get us mentally prepared for any future computerized tests.
2. Security is achievable. Most of the students surveyed agreed that the tests were secure, and cheating was controlled.
In this case study, the Foundations of Nursing course still displayed all the test items on one screen, but questions and answer options were all scrambled. Therefore, students who were sitting next to each other did not have the same question order. Furthermore, test security was enhanced by using the Angel Secure Browser, which disables non-test-related clicks, and by using passwords, so that only the students in the classroom were allowed to access the examination.
The rest of the nursing courses decided to leave questions displayed one at a time, and no backtracking was allowed. Together with the presence of two proctors, cheating was reduced to the minimum. Student B voiced his/her opinion in the survey:
We recently took a quiz and we were given one question at a time and proceeded to the next question without the chance of going back, I like that way of administering the questions because everyone gets a different question at a time on the screen and it is much harder for students to cheat. It also helped me to better focus on the one question that I was currently working on instead of continually getting distracted with the next question.
3. It is convenient for both students and instructors. Most of the students liked the fact that they could see their examination grades right after the examination. Student C said the following in the survey:
I like taking the tests and quizzes on my computer because I like to know my grade right afterwards instead of waiting a long time for the teachers to hand grade them.
Interviews with the instructors showed that they liked the powerful regrading feature offered by Blackboard-Angel LMS; with a few button clicks, the instructor could regrade a problematic question and apply it to all students within seconds. Students also were in favor of it because they could get their grades updated instantly.
Both the instructors and students were in favor of the individualized test review feature. Most of the students agreed that it was very efficient to go directly to the questions they missed and skip those questions that they already knew. Here is what student D described in the survey:
I like the individualized test review because it saves time and everyone can see what questions they got wrong and thus could ask instructors questions.
4. It provides students hands-on practice. Computer-related testing anxiety is a common topic in nursing education. In this case study, about half of the students surveyed agreed that this computerized testing helped them overcome computer-related testing anxiety. Student E wrote the following comment in the survey:
I was afraid of computerized test in the beginning, but because of the preparation the class provided I am now in favor of it and will make it easier when I take the NCLEX-RN exam later.
However, some of the students believed that the LBT did not help ease their computer-related anxiety at all. They felt more stressed by taking the examinations on their laptops. One student expressed his/her concern in the survey:
It has been very difficult for me to get used to the computerized testing program. I got more anxiety and made many mistakes using the computer mouse.
For those students who experienced difficulties with the laptop, further coaching and practice can be offered. For example, a wireless mouse can be recommended to reduce the mouse-related glitches of selecting wrong options. Another strategy is to provide more practice opportunities. For example, in the Foundations of Nursing course, the lead instructor administered a quiz on the laptop before his lecture every week. As the old saying goes "practice makes perfect," and computer-related anxiety will decrease with hands-on practice on the computer.8 Another student echoed the same belief with some hope in the survey:
I still have anxiety on computerized testing, but it is natural to some extent. I need to get used to this type of test for the exam. Hopefully as the program progresses my computerized testing anxiety will diminish gradually.
5. Continuous technical support is the key. Glitches are inevitable with any technology-based program. Most of the students reported no major glitches during the first two secured examinations. Most of the glitches were related to the wireless network and laptop operating system. Continuous and consistent technical support is crucial to the success of this testing program. In this case study, more than half of the students agreed that if glitches did happen, the support team was responsive and could usually fix the problem right on the spot. Student F described his/her experience in the survey:
I had a virus on my laptop on the first exam day and couldn't run the Angel Secure Browser, the tech person immediately gave me a loaner laptop and I was able to take the exam with the rest of the class.
The college has two full-time instructional technologists who are responsible for secured testing. They make sure to be present at every secure examination, especially during the first 10 minutes when most of the glitches happen. In addition, the two technical support persons are given two extra loaner laptops just in case some students forget to bring their laptops or the glitches cannot be fixed right on the spot.
The secure LBT model is summarized in Figure 2. Four major elements (laptop initiative, robust LMS, wireless campus, and continuous support) are the pillars for the success of secure LBT. The laptop initiative ensures that every student has the same brand of laptop that meets the minimum requirements of conducting tests. All the tests are designed, developed, and hosted within a robust LMS (such as Blackboard-Angel). The secure browser feature also comes with the LMS. The authors' entire college campus has wireless Internet services available at zero cost to all enrolled students. This service allows them to access the tests through the LMS. Lastly, consistent technical support to instructors and students is crucial to the continuous success of the LBT program.
The pros and cons of the LBT model are further compared with the traditional computer laboratory model in Table 4.
It seems that security of the LBT program can be achieved with proper training and continuous support. Numerous benefits of LBT can be reaped if implemented properly.
A limitation of this study is that the findings are not generalizable to other universities or colleges, especially settings outside North America for two reasons: (1) purposive convenience sampling was used, and (2) different settings may use different IT resources, such as different LMS. However, merits of LBT that emerged from this study provide strong rationales to implement computerized testing. Given adequate IT resources and adjustment of LMS, the LBT model described in this study may be transferable to other settings. For example, many campuses around the world are using Blackboard Learn as LMS, and the Respondus Lockdown Browser might then be considered to achieve similar LBT program.
Although Reising,7 in 2003, pointed out that there were no significant differences in NCLEX-RN passing rates between the students who were exposed to computer-based testing in their nursing programs and those who were not exposed, the same study noted that the increased passing standard in 1998 might have caused a decline in passing rates during that period and thus influenced the study results. Further studies of the relationship between LBT within nursing programs and the passing rate of students' nursing board examinations are recommended.
1. Halkitis PN, Leahy JM. Computerized adaptive testing: the future is upon us. Nurs Health Care. 1993; 14 (7): 378-385. [Context Link]
2. Bloom KC, Trice LB. The efficacy of individualized computerized testing in nursing education. Comput Nurs. 1997; 15 (2): 82-88. [Context Link]
3. Bugbee AC. The equivalence of paper-and-pencil and computer-based testing. J Res Comput in Educ. 1996; 28 (3): 282-299. [Context Link]
4. EduTools. http://www.edutools.info/item_list.jsp?pj=8. Accessed in July 2011. [Context Link]
5. Olsen J. Applying computerized adaptive testing in schools. Meas Eval Counsel Dev. 1990; 23: 31-38. [Context Link]
6. Anna D. Computerized testing in a nursing curriculum: a case study. Nurse Educ. 1998; 23: 22-26. [Context Link]
7. Reising DL. The relationship between computer testing during a nursing program and NCLEX performance. Comput Inform Nurs. 2003; 21 (6): 326-329. [Context Link]
8. Jacobs P, Koehn ML. Implementing a standardized testing program: preparing students for the NCLEX-RN. J Prof Nurs. 2006; 22 (6): 373-379. [Context Link]
9. Vrabel M. Computerized versus paper-and-pencil testing methods for a nursing certification examination: a review of the literature. Comput Inform Nurs. 2004; 22 (2): 94-98. [Context Link]
10. Fuszard B. Innovative Teaching Strategies in Nursing. Gaithersburg, MD: Aspen Publishers Inc; 1999. [Context Link]
11. Moskal P, Caldwell R, Ellis T. Evolution of a computer-based testing laboratory. Innovate J Online Educ. 2009;5(6). [Context Link]
12. Dibartolo MC, Seldomridge LA. A review of intervention studies to promote NCLEX-RN success of baccalaureate students. Comput Inform Nurs. 2008; 26 (5): S78-S83. [Context Link]
Back to Top