Educators have promoted inquiry, teaching through questioning, to engage students dynamically in the attainment and retention of knowledge and skills. Traditionally, students have assumed passive roles, observing the instructors' performances, rather than actively participating with instructors and peers.1-4 Often, lectures present new knowledge, but lack dedicated time for teaching higher-order thinking skills such as analysis, synthesis, and problem-solving.1-4 Inquiry may be used to teach these higher-order thinking skills and increase student participation.4 However, encouraging student participation in large-lecture courses can be daunting due to time constraints and the reticence of pupils to ask or answer questions in front of large groups.

Audience response system (ARS) technology can address this need for increased interactivity and critical thinking in a large-classroom environment.1,2,5 Students' use of an ARS may promote student comprehension and retention by offering each learner several chances to participate in class, anonymously answering questions posed by the instructor. Researchers have suggested that students are satisfied with the use of ARSs, but the research is mixed on whether the use of these systems actually increases test scores. This study sought to investigate whether students' use of an ARS to answer multiple-choice PowerPoint (MCPP) slides (Microsoft, Redmond, WA) is more effective in increasing test scores than using MCPP slides alone.

AUDIENCE RESPONSE SYSTEMS

An ARS is a communication technology that allows participants to interact with the lecturer; many of these systems seamlessly merge with PowerPoint presentations.2 The lecturer may post multiple-choice or true-false questions on a PowerPoint slide, and then each student answers by pressing an individual keypad button corresponding to his/her answer. These keypads, or clickers, use infrared, radio frequency, Bluetooth, or Wireless Fidelity (Wi-Fi) technology to communicate with a receiver, which is plugged into a universal serial bus (USB) port on the podium computer.1,6,7 After polling the audience, a bar graph of the classroom responses is displayed on a projected slide. The graph allows the presenter to gauge the level of participant understanding immediately and discuss the correct and incorrect answers.3 The lecturer may choose to review previous content, tailoring the lecture to comprehension level.1,2,8-15 The responses of each individual may be saved and then later printed in a variety of report formats.16,17 Thus, the responses to the ARS questions may be compared with future test scores if desired.

As new communication technologies become available, ARSs continue to advance. Currently, radio frequency is preferred to infrared technology because it accommodates larger audiences with less interference and does not require that the keypads be aimed at the USB receiver while inputting answers.1,16,17 Enterprising researchers are experimenting with using PDAs to communicate with the audience response receivers.1,7 In addition, student terminals or full laptop keyboards may allow participants to enter their answers to open-ended questions. The instructor may randomly select a student's typed response to project on the slide.18 Some students, however, may find laptops too cumbersome to tote to class and too slow to activate.19 Thus, laptop use in large lecture-class environments may be seen as burdensome by students.

The use of ARSs may increase active learning by students in large lecture classes5,13,20 and in small groups.1,3,6,21,22 When inserting MCPP slides within a lecture, other content slides must be omitted to stay within the lecture's allotted period. In addition, the lecturer must allow more time for each MCPP slide to present the question, discuss the correct and incorrect answers, and possibly reteach the content area. Thus, the use of an ARS may decrease the amount of lecture content that may be addressed.1,6,9 Increased student comprehension may, however, outweigh this time limitation. In addition, increased lecture preparation time is required to develop questions that encourage application, synthesis, and problem solving.3

Students, as well, have said that using the ARS helps them to judge their understanding of the content2,8,11,15 and provides them the safety of answering anonymously.1,2,6,9,11,23-25 Many students believe that the ARS questions help them perform better on quizzes and examinations.2,13 Stein et al13 stated that 89% of nursing students perceived a beneficial effect of discussing distracters, citing increased comprehension of the lecture content. In addition, the ARS questions may assist students in becoming familiar with the lecturer's test-question formats.13 Students may have "emotional" investments in their responses, which increase their attentiveness6 and participation.5,10,14 Lastly, the use of ARS positively influences class attendance.6,22,26 Overall, this technology has been widely embraced by students.2,6,8,11,17,20,27,28 This may be due to the partitioning of lecture content into small, manageable units and allowing students intermittent breaks from the lecture while increasing student engagement and attention.3,6,11-13,27,29 DeBourgh2 suggested that ARS questions be used every 20 minutes to prevent "cognitive overload."2^(p81) Miller et al12 demonstrated that students assign higher lecturer evaluation scores to instructors who use an ARS.

Some researchers, however, have suggested that satisfaction levels are greater in the freshman-sophomore level courses than junior-senior level courses.20 They suggest that students may appreciate an ARS more when studying an unfamiliar topic. Some students have registered dissatisfaction with the use of an ARS in taking attendance,6 and others resent the increased expectation to participate in class.2 Of course, students have been known to deceive,6 responding with multiple keypads on quizzes and attendance.

Although students are generally satisfied with audience response technology, the literature is mixed on whether the students' use of ARSs increases student performance on test scores. Some researchers have demonstrated clear increases in test scores in ARS-enhanced lecture courses rather than typical lecture courses.19,22,28-30 Crossgrove and Curran20 found that students performed significantly better on test questions that had been discussed previously in ARS-enhanced lectures. However, retention of course material may be greater in introductory versus junior-senior level courses. Preszler et al26 discovered that the test scores improved as the lecturer's use of ARS questions increased. Other research showed no difference in test scores between courses utilizing didactic lectures versus lectures that incorporated ARS.11-13

The conflicting results on ARSs' ability to positively affect test scores could be attributed to how closely ARS questions parallel test content, the frequency of MCPP slides within the lecture, differences between freshman-sophomore and junior-senior course content and student expectations, the instructors' experience with inquiry, or other confounding variables, such as student age. In addition, the lecturer may have already incorporated inquiry or small group work into his/her course; thus, one more method of encouraging student engagement, ARS, may not result in further statistically significant score increases.20 Although ARSs may increase scores in traditional, didactic courses with one-way communication, this technology may not add any advantage in courses that already engage the learner in two-way communication. Ultimately, the interaction between pedagogy and ARS use defines learning outcomes,9 and the proposed benefits of ARSs must justify institutional and student keypad costs.3

Use of Audience Response Systems in Nursing Education

Nursing researchers are beginning to investigate the use of ARSs to augment the acquisition and retention of nursing knowledge by nursing students. Three nursing studies evaluated the impact of ARSs on course test scores8,11,13; only the study by Abdallah8 showed increases in test scores. In 1995, Halloran11 investigated the use of computer-managed instruction (CMI) and keypads in a medical/surgical course taught to baccalaureate junior nursing students. The experimental group (14 students) used CMI and keypads in the lectures, whereas the control group (14 students) was taught the same content by the traditional lecture format, including discussion, overheads, and oral questioning. Both groups took the same three multiple-choice tests. The differences in test scores between groups were not significant; no differences in test scores were found between those nursing students who used CMI and those who participated in traditional lectures.

Stein et al13 investigated the use of an ARS for pretest reviews in two successive freshman nursing anatomy and physiology courses (155 and 128 students, respectively). The students received three ARS test reviews and one traditional, lecture-format review prior to the examinations in each course. The ARS test review used a Jeopardy! game format (a television game show in which contestants are presented with answers and must respond with the correct question) with 25 questions. The question formats included multiple-choice questions, true-false questions, and labeling of diagrams. The 92% of students voiced positive satisfaction with the ARS technology. Most of the students (94%) perceived a beneficial effect on their test scores, although the test averages were not significantly higher in tests that followed an ARS review. If the students answered poorly on the ARS question in class, they generally gave significantly more correct responses on a similar test question when it was on the examination. Eighty-nine percent of the students felt that discussing the distracters, or wrong answers, to the ARS questions assisted their comprehension of the lecture content.

In fall of 2006, Abdallah8 investigated the effects of using a personal response system (PRS) with 71 junior nursing students in a nursing foundation course. Students were awarded extra participation points if they used their clickers in all classes. Initial and midterm assessments of the PRS were completed. Students' exposure to PRS questions resulted in 70% or more students correctly answering the same or similar content questions on the course quizzes; 80% to 92.5% answered these questions correctly on the midterm. The higher percentage of students answering correctly on the midterm may be due to the students' realizations that the quiz questions came from the PRS slides. At midterm, 67% of the students felt that the PRS assisted them in understanding the lecture content, as compared with 56% on the first day of class. Thus, clearer understanding of what constituted core content may have led to higher test scores, rather than PRS use. Studies are needed to separate the effects of active participation from that of ARS use.

Lastly, DeBourgh2 conducted an online survey of 92 advanced nursing therapeutics course students on their satisfaction levels with ARS use. Sixty-five students completed the anonymous questionnaire 1 week before the end of the semester. Most of the responding students (75.8%) recommended continued use of clickers, and 66% believed that the ARS increased their quiz and examination scores. Fifty-three percent came to class "better prepared."2^(p83) DeBourgh2 did not evaluate the impact of ARS use on test scores.

PURPOSE

Subjects in the ARS group were significantly older than the participants in the control group (mean, 27.8 [SD, 9.4] years and 21.4 [SD, 3.0] years, respectively, P < .0001) (Table 1). The subjects in the ARS group were more likely to have prior degrees than those in the MCPP group, reaching a significance level of .007. In addition, the ARS group had more prior ARS course experience than the subjects in the MCPP group (P = .036). No significant differences were found between groups for sex, ethnicity, number of text messages per week, and prior experiences with ARS educational offerings, healthcare employment, and informational technology employment.

Table 1 Demographics by ARS and MCPP Groups

Participant Test Scores

No significant differences between groups were found for the overall test scores for examination 2, 3, or 4. The P values were .475, .326, and .207, respectively (Table 2). Likewise, no subtest differences were found between the groups.

Table 2 Median Difference of Test Scores by Group

Satisfaction Questionnaire Responses

Differences in percentages between the control and ARS groups on the satisfaction statements about MCPP slides are presented in Table 3. The answers differed significantly between groups for two statements. The control group was more likely to agree that correct MCPP answers corresponded to correct examination answers (statement 5): 88.9% for the control and 66.7% for the ARS groups (P = .003). However, since the control group's answers to the MCPP slides were not collected, the accuracy of their perception cannot be verified. The ARS group was more likely to disagree that incorrect MCPP answers corresponded to incorrect examination answers (statement 6), 74.4% versus 57.4% (P = .041). The ARS group had performed better on the test questions than on the MCPP questions. The ARS and control groups both agreed that (1) MCPP slides increased understanding of the lecture content (statement 2): 98.7% and 100%, respectively; (2) a discussion of the correct answers to the MCPP slides increased understanding (statement 3): 98.7% and 98.1%, respectively; and (3) a discussion of the incorrect answers to the MCPP slides increased understanding (statement 4): 97.4% and 98.1%, respectively.

Table 3 Satisfaction Questionnaire Responses by Group (Statements 1-23)

Interestingly, the control group overwhelmingly disagreed with two statements pertaining to ARS use (Table 3). The control subjects rejected statement 12, which declared that they wished they had the opportunity to use the ARS; 72.2% said no. In addition, 77.8% rejected the statement that they would choose an ARS course over a non-ARS course.

Only the intervention group predominantly agreed that ARS use was beneficial (Table 3). The students felt that ARS feedback increased comprehension of the material (93.6%) and increased retention of lecture content (89.7%). Furthermore, the students in the ARS group agreed that the use of an ARS increased their examination scores (64.1%) and did not decrease their scores (97.4%). Most were glad to have the opportunity to use the ARS (94.9%) and would recommend that the ARS be used in nursing pharmacology courses (92.3%) and in other nursing courses (85.9%). In fact, 61.5% of the interventional students would choose an ARS course over a non-ARS course. Because junior-senior students were noted to be less embracing of ARS technology,20 the authors expected that students with prior degrees might not concur; this was not the case. Interestingly, 64.1% of the ARS group would not choose a course that used laptops to answer MCPP questions. Similarly, 83.3% of the respondents would prefer to use clickers versus laptops in answering MCPP questions.

DISCUSSION

Baccalaureate nursing students who attended nursing pharmacology lectures using MCPP slides and an ARS did not demonstrate increased comprehension and retention of pharmacological knowledge compared with those students relying solely on MCPP slides. There were no significant differences between course test scores or subtest scores between the MCPP and the MCPP plus ARS groups. The ARS group, who had older students with more prior degrees, performed the same as the control group students, who were younger. These results suggested that the use of an ARS did not provide an additional boost to students' test scores, and these results are similar to the findings of Stein et al13 and Halloran.11 The test scores may have been influenced more by the MCPP slides, which increased student engagement in both the ARS and control groups. Thus, MCPP use in both groups may have encouraged active inquiry and positively affected both groups' test scores, resulting in no discernable differences in test scores between groups, as Crossgrove and Curran suggested.20

As Stein et al13 observed, the students believed that ARS use enabled them to comprehend course material and attain higher test scores, even though the test scores were not significantly increased. Similarly, Abdallah8 noted that students felt that the ARS assisted their comprehension of lecture content, and DeBourgh2 remarked that students positively attributed ARS use to achieving higher examination scores. In the current economic climate, the lack of improvement in test scores may outweigh the increased student satisfaction levels when weighing the financial costs of ARSs to the institution and students. However, the merits of obtaining immediate student feedback by the instructor must not be underestimated when concurrently tailoring a lecture to their needs.

Although the test scores were not significantly different, the histograms of student responses allowed the instructor to tailor the lectures to overall student needs, instead of the vocal few. If instructors use existing ARS technology already established at their institution, they may decrease student costs by requiring one clicker for all courses at that institution. In addition, favorable student responses suggest that this technology improves the educational climate, which may be reflected in students' evaluations of faculty.12 Furthermore, if the control group had been allowed to try the ARS, the students might have embraced this technology.

1. Collins LJ. Livening up the classroom: using audience response systems to promote active learning. Med Ref Serv Q. 2007;26(1):81-88. [Context Link]

2. DeBourgh GA. Use of classroom "clickers" to promote acquisition of advanced reasoning skills. Nurse Educ Pract. 2008;8(2):76-87. [Context Link]

3. Dufresne RJ, Gerace WJ, Leonard WJ, Mestre JP, Wenk L. A classroom communication system for active learning. J Comput High Educ. 1996;7(2):3-47. [Context Link]

4. Kraft RG. Group-inquiry turns passive students active. Coll Teach. 1985;33(4):149-154. [Context Link]

5. Lowry PB, Romano NC, Guthrie R. Explaining and predicting outcomes of large classrooms using audience response systems. Proceedings of the 39th Hawaii International Conference on System Sciences, January 4-7, 2006. Los Alamitos, CA: Computer Society Press; 2006. [Context Link]

6. Caldwell JE. Clickers in the large classroom: current research and best-practice tips. CBE Life Sci Educ. 2007;6(1):9-20. [Context Link]

7. Menon AS, Moffett S, Enriquez M, Martinez MM, Dev P, Grappone T. Audience response made easy: using personal digital assistants as a classroom polling tool. J Am Med Inform Assoc. 2004;11(3):217-220. [Context Link]

8. Abdallah L. Reflective teaching with technology: use of a personal response system and publisher's web site to enhance students' performance in a nursing assessment and skills course. Online J Nurs Inform [electronic resource]. 2008;12(1). http://ojni.org/12_1/abdallah.html. Accessed March 16, 2009. [Context Link]

9. Cain J, Robinson E. A primer on audience response systems: current applications and future considerations. Am J Pharm Educ. 2008;72(4):77. [Context Link]

10. Gamito EJ, Burhansstipanov L, Krebs LU, Bemis L, Bradley A. The use of an electronic audience response system for data collection. J Cancer Educ. 2005;20(1 suppl):80-86. [Context Link]

11. Halloran L. A comparison of two methods of teaching. Computer managed instruction and keypad questions versus traditional classroom lecture. Comput Nurs. 1995;13(6):285-288. [Context Link]

12. Miller RG, Ashar BH, Getz KJ. Evaluation of an audience response system for the continuing education of health professionals. J Contin Educ Health Prof. 2003;23(2):109-115. [Context Link]

13. Stein PS, Challman SD, Brueckner JK. Using audience response technology for pretest reviews in an undergraduate nursing course. J Nurs Educ. 2006;45(11):469-473. [Context Link]

14. Uhari M, Renko M, Soini H. Experiences of using an interactive audience response system in lectures. BMC Med Educ. 2003;3:12. http://www.biomedcentral.com/1472-6920/3/12. Accessed March 16, 2009. [Context Link]

15. Johnson JT. Creating learner-centered classrooms: use of an audience response system in pediatric dentistry education. J Dent Educ. 2005;69(3):378-381. [Context Link]

16. Barber M, Njus D. Clicker evolution: seeking intelligent design. CBE Life Sci Educ. 2007;6(1):1-8. [Context Link]

17. Kelley KA, Beatty SJ, Legg JE, McAuley JW. A progress assessment to evaluate pharmacy students' knowledge prior to beginning advanced pharmacy practice experiences. Am J Pharm Educ. 2008;72(4):88. [Context Link]

18. Foegen A, Hargrave CP. Group response technology in lecture-based instruction: exploring student engagement and instructor perceptions. J Spec Educ Tech. 1999;14(1):3-17. [Context Link]

19. Nosek TM, Wang W, Medvedev I, Wile MZ, O'Brien TE. Use of a computerized audience response system in medical student teaching: its effect on active learning and exam performance. In: Reeves T, Yamashita S, eds. Proceedings of World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education. Chesapeake, VA: AACE; 2006:2245-2250. [Context Link]

20. Crossgrove K, Curran KL. Using clickers in nonmajors- and majors-level biology courses: student opinion, learning, and long-term retention of course material. CBE Life Sci Educ. 2008;7(1):146-154. [Context Link]

21. Hoopes LL. Educator highlight: Morris Maduro. CBE Life Sci Educ. 2008;7:3-4. [Context Link]

22. Freeman S, O'Connor E, Parks JW, et al. Prescribed active learning increases performance in introductory biology. CBE Life Sci Educ. 2007;6(2):132-139. [Context Link]

23. Elliot C. Using a personal response system in economics teaching. Int Rev Econ Educ. 2003;1(1):80-86. [Context Link]

24. Holmes RG, Blalock JS, Parker MH, Haywood VB. Student accuracy and evaluation of a computer-based audience response system. J Dent Educ. 2006;70(12):1355-1361. [Context Link]

25. Torbeck L. Enhancing programme evaluation using the audience response system. Med Educ. 2007;41(11):1088-1089. [Context Link]

26. Preszler RW, Dawe A, Shuster CB, Shuster M. Assessment of the effects of student response systems on student learning and attitudes over a broad range of biology courses. CBE Life Sci Educ. 2007;6(1):29-41. [Context Link]

27. Latessa R, Mouw D. Use of an audience response system to augment interactive learning. Fam Med. 2005;37(1):12-14. [Context Link]

28. Pradhan A, Sparano D, Ananth C. The influence of an audience response system on knowledge retention: an application to resident education. Am J Obstet Gynecol. 2005;193:1827-1830. [Context Link]

29. Conoley J, Moore G, Croom B, Flowers J. A toy or a teaching tool? The use of audience-response systems in the classroom. Techniques. 2006;81(7):46-48. [Context Link]

30. Schackow TE, Chavez M, Loya L, Friedman M. Audience response system: effect on learning in family medicine residents. Fam Med. 2004;36(7):496-504. [Context Link]

For more than 19 additional continuing education articles related to education, go to http://NursingCenter.com\CE.