J Dent Educ. 72(9): 1042-1047 2008
© 2008 American Dental Education Association
Educational Methodologies |
Electronic Voting in Dental Materials Education: The Impact on Students’ Attitudes and Exam Performance
Michele E. Barbour, M.Phys., Ph.D., P.G.C.H.E.
Key words: dental materials, biomaterials, lectures, educational techniques
Submitted for publication 02/04/08;
accepted 04/21/08
 |
Abstract
|
|---|
Dental materials is an integral part of any undergraduate dental curriculum and is most commonly taught in a traditional didactic, lecture-based format. It suffers from the ignominy of being viewed by many as a dry, factual subject with little to excite or engage the student. In this article, the author presents the experimental use of an electronic voting (eVoting) system in an undergraduate dental materials course. The practical and aspirational aspects of its application are described. The objective was to assess the student perception of the experiment and the impact on end-of-course examination results. The eVoting system proved overwhelmingly popular with the students with 95 percent in favor of its use at the beginning of the course and 91 percent at the end. There was, however, no statistically significant impact on the results of the examination at the end of the course, when compared to the previous year’s cohort of students for whom eVoting was not used. eVoting encouraged student interaction and engagement and contributed to student satisfaction but was not seen to affect the outcome measurement (end-of-course examination result).
Dental materials is an integral part of any undergraduate dental curriculum. It is imperative that students in training and newly quali-preparation, properties, and limitations of currently available dental materials and the wherewithal to apply critical reasoning to materials that will be subsequently developed and become available throughout their professional careers. It is unfortunate, then, that dental materials is sometimes seen as a rather dry subject, and many students associate materials courses with memorizing tables of facts and figures.
It is widely recognized that lectures have certain disadvantages with regards to the promotion of deep and long-lasting understanding of a topic.1–3 Nevertheless, the majority of formal, didactic dental education is conducted in lecture format.4 In terms of staff resources, lecturing is the most efficient way to teach large groups of students, and is often the only practical option for substantial parts of the curriculum, given the numbers of faculty available to teach certain topics and the configuration of classroom space within dental schools.3 Rather than denigrate the lecture format, therefore, it is constructive to consider means by which lectures can be modified and developed to better support the students’ learning process.
In this article, the use of an electronic voting (eVoting) system, or audience response system (ARS), in undergraduate dental materials education at a UK dental school is described. Such systems have been applied in situations in which the lecturer or teacher wishes to encourage the participants to take a more active role in an otherwise passive teaching format such as a lecture.5 One of the earliest discussions of the use of these systems in higher education was an editorial by Turpin,6 who describes the use of an ARS at a meeting of professional orthodontists. Since that time, eVoting has been applied in medical7,8 and nursing9,10 education and, recently, in dental education in courses on pediatric pulp therapy11 and in an experiment using first-year dental students.12
This article describes the use of eVoting in a course entitled "Basic Biomaterials Science," which is delivered to second-year undergraduate dental students and which serves the broad purpose of introducing students to the concepts and scope of dental materials science and investigating some of the most common restorative materials. The practical application of the technology, the aspirations that lead to its use, the students’ opinions of the experiment, and the impact on the associated end-of-course assessment are reported.
|
Materials and Methods
|
|---|
Two consecutive cohorts of second-year dental students at the University of Bristol Dental School are discussed in this article. There were sixty-seven students in the 2006 cohort and seventy-five students in 2007. The male-female ratios were approximately equal in the two cohorts (55 percent female in 2006, 57 percent female in 2007), and there were more mature students in 2007 (zero in 2006, seven in 2007), a mature student being defined as age twenty-one or over at registration for the degree course.
The Basic Biomaterials Science course consisted of ten lectures, four practical classes, a problem-based learning project, two visits to dental practice, nine progress tests, and a final end-of-course examination. The practical classes comprised sessions wherein the students were taught to handle common dental materials and assess various properties (hardness, compressive strength, detail reproduction) as a function of material composition and handling.
The examination was comprised of fifty multiple-choice and ten short answer questions. The multiple-choice questions were all in a true/false format and were negatively marked: a correct answer scored +1 point, an incorrect answer scored -1 point, and no answer scored 0. The short answer questions were not negatively marked and were each scored out of 5 points (credit was awarded for correct answers but not deducted for incorrect answers).
The same examination paper was administered to both cohorts of students, with the exception of two multiple-choice and two short answer questions, which were subject to minor alterations. The questions that were changed were excluded from the analysis. It is not common practice to reuse exams, and this would not have been expected by the students. Furthermore, the answers were completed on the examination paper, and individually named papers were distributed; thus, it was possible to ensure that no examination paper left the classroom. It can therefore be concluded that the 2007 cohort had no knowledge of the 2006 paper, nor had they any expectation that this information would be useful even if they could have obtained it. The students were encouraged to think of the weekly progress tests as practice for the exam since the questions were in the same format. The questions on the progress tests were changed between 2006 and 2007; therefore, the results are not directly comparable and will not be discussed further.
An ARS using individual hand-held voting devices was utilized with supporting software (TurningPoint version 3.1.7.620
[EC]
6; Turning Technologies, Ohio, USA), interfaced with Microsoft PowerPoint in selected lectures delivered to the 2007 cohort of students. The ARS was used by the author in a total of six of the ten lectures; three lectures were delivered by other members of staff using traditional didactic methods; and one lecture was delivered by the author using a "giant tutorial" format. By contrast, in 2006, nine lectures were delivered by the same members of the staff using traditional didactic methods and one with the giant tutorial format as in 2007. The giant tutorial was run by assigning the students open-ended questions such as "Properties of a fantasy dental material" and "What factors make the oral environment a materials scientist’s worst nightmare?" and asking them to discuss the questions in groups of three to five and come up with answers; their reports were followed by feedback to the lecturer and collation of the results on a flipchart. The students were asked, as in a tutorial, to explain their suggestions and to offer elaboration or constructive criticism on their peers’ suggestions as well as defend their own contributions.
Individual hand-held polling devices were distributed to each student, and they were shown how and when to use these devices to cast a vote, including how to override a previous answer if they changed their mind. The ARS integrates into Microsoft PowerPoint such that questions, answers, and statistics are displayed side-by-side in real time during the lecture.
eVoting slides were incorporated into groups at two or three points, approximately equally spaced, during the lecture. In the six lectures using eVoting, there were a total of six, eight, nine, nine, twelve, and fifteen eVoting questions. Questions were either true/false or, more commonly, had between three and seven potential answers for students to analyze prior to selecting one best response. Questions were directly related to the lecture material that had been communicated immediately prior to presentation of the questions. One or two simple, more factual questions were often used at first to encourage the students to engage with the activity; not surprisingly, these questions had a high proportion of correct answers since the relevant facts had been imparted only a few minutes previously. After the factually oriented items, more difficult questions were presented that required students to comprehend the concepts in the lecture and apply this new understanding to a unique situation. The later group of sophisticated questions were the more educationally rewarding questions with respect to monitoring the students’ learning and encouraging the students to think critically and deeply rather than focusing on simple factual recall.
Feedback was collected twice during the term specifically with regard to the students’ perception of eVoting, as distinct from the student feedback always collected as part of this course. The students were asked "How do you like the eVoting system?" and invited to select a response, using the eVoting handsets, from a modified Likert scale consisting of "I love it!," "I quite like it," "I’m ambivalent about it," "I don’t like it much," or "I hate it!" Students were also invited to comment on the eVoting system, as well as other aspects of the course, at the end-of-term feedback session.
|
Results
|
|---|
The marks for the total examination, separated according to question type and question number, are shown in Table 1
. The p values, as determined using unpaired t tests, are also indicated. Statistical significance was set at 
=0.05. There was no statistically significant difference between the total examination results, or the mean results for the two sections (multiple-choice and short answer), for the students in the 2006 and 2007 cohorts. Within individual questions, however, there were some significant differences—two in which the mean mark increased in 2007 in comparison to 2006, and two in which the converse was true.
The outcome of the question to assess student feedback on eVoting is shown in Figure 1. Feedback collected near the beginning of the lecture course, after the first eVoting lecture, indicated that 95 percent students responded with either "I love it!" or "I quite like it," with the majority (62 percent) in the former category. Feedback collected at the end of the course showed a similar reaction, with 91 percent responding with either "I love it!" or "I quite like it," and the majority again (67 percent) in the former category, although there was an increase in the number of students responding with negative feedback ("I hate it!" 7 percent). During the general-purpose feedback session students were also encouraged to write discursive comments about any aspect of the course, and many individuals commented that they enjoyed the use of the eVoting system and would like to see it rolled out to other lectures.
View larger version (13K):
[in this window]
[in a new window]
|
Figure 1. Students’ feedback regarding their opinion of the use of the eVoting system obtained at the beginning of the course (after the first eVoting lecture) and at the end of the course (after the final eVoting lecture)
|
|
|
Discussion
|
|---|
eVoting has been used as a teaching and learning aid in a number of disciplines, although medicine seems to dominate the field to date. In a randomized trial at the University of Adelaide, it was concluded that incorporation of eVoting into lectures on cancer screening offered no advantage in terms of test scores, and was thought by the lecturers to render lectures more difficult to prepare than traditional styles.8 On the other hand, the same study concluded that eVoting did encourage students to take an active role in the lecture, and lectures in which eVoting were used ranked higher in student feedback questionnaires. Another study of eVoting in tutorials concluded that long-term retention of information was slightly increased by the use of the technology and that there was an increase in student-student interaction, albeit at the expense of student-tutor interaction.7
In the study discussed here, the ARS was used for two main purposes, in addition to facilitating students’ learning: to assess the students’ comprehension in real time (during the lecture as it was being conducted), and to maintain high levels of student concentration and engagement by providing regular changes of pace and encouraging the students to take an active role in using the information communicated during the lecture. With respect to assessing students’ understanding in real time, when it became apparent that a significant proportion of the students were struggling with a certain concept or topic based on their ARS answers, the instructor used this opportunity to immediately revisit that material and clarify any confusion, using alternative explanations, discussions with students, and a "chalk and talk" methodology.13 When it was clear that the vast majority of students had grasped the fundamentals, the course instructor was able to proceed with the lecture without laboring a point unnecessarily.
The ARS proved very popular with the undergraduate students (Figure 1
), with 95 percent of students in favor of its use at the beginning of the course and only slightly less, 91 percent, at the end. This is substantially higher than the typical student approval rating of eVoting systems of around 70 percent, reported in a review by Caldwell5 and slightly higher than the 88 percent reported in a study of dental students by Holmes et al.12 The anecdotal feedback obtained during the feedback session was also very positive, with many students suggesting that the ARS could profitably be used in other lectures. The similarity of the feedback at the beginning and end of the course indicates that the initial popularity was probably not associated with "novelty value"—or, if it was, the novelty had not worn off after a term’s teaching. A few students gave a strong negative response to the eVoting: 1 percent at the beginning of the course and 7 percent at the end. None of the students gave written or oral feedback as to what it was they did not like about the eVoting; all written comments were positive. It is conceivable that some students simply do not like to interact in lectures: didactic teaching is still pervasive in modern education, and a straw poll indicated that few of the students had experienced interactive lectures before. It can not be concluded that this is the reason that some students did not like eVoting, but in future years the author will investigate this further.
Interactive techniques such as eVoting are generally considered good practice and are associated with greater student engagement and enhanced and more sophisticated learning, compared with non-interactive lectures.14–16 eVoting offers one major advantage over other interactive lecture techniques, however, in that all students can take part and that students may present their opinions anonymously. Although the author experiences good levels of interactions with a significant proportion of the student body once they have become accustomed to an interactive lecture format, there will always be a subset of students who, for one reason or another, do not feel comfortable "shouting out" their answers in the presence of their peers, and eVoting gives these students a voice and an equal footing with the more confident students, as noted by Caldwell.5 In future years it would be valuable to probe the students further on their perception of the usefulness of eVoting (for example, to ask the students whether its use motivated them to participate more in the lecture) and to determine whether they felt that it promoted deep learning and comprehension or simply made the lectures more stimulating.
Overall, the use of the ARS did not have any statistically significant, positive or negative, effect on examination results. Care was taken to ensure that all other aspects of the course were the same in the two years, so that changes in performance could, as far as possible, be attributed to the use of this technology. There were some changes in performance in individual short answer questions (Table 1), but since two questions were answered better in 2007 and two worse, it is likely that this is a result of simply random variation between the year groups rather than any effect of learning technology. Although it may be considered preferable to perform a longitudinal study whereby the same group of students is assessed, for example, before and after an intervention, this was not feasible in the study presented here. Another potential route for development of this technique would be to use the eVoting system as an introduction to small-group work within the lecture setting and to poll answers from the groups after such group discussions had taken place.
It should be noted that one limitation of this study is the sample size: two cohorts of students were compared. This was done for professional reasons: once the ARS became available, it was deemed desirable to incorporate it into the teaching as soon as possible rather than wait for several years to provide control students for a study such as this. It is possible that reflection on the use of the eVoting system over a greater number of years, and successive cohorts of students, will provide further insight into the effects on medium- and long-term learning and comprehension.
The ARS was simple to set up and operate, and in contrast to Duggan et al.,8 the instructor did not find the extra work required to prepare lectures in this format excessively onerous. There are now approximately seven commercially available systems for eVoting in academic settings; a review of their strengths and weaknesses can be found elsewhere.17
|
Conclusion
|
|---|
Use of eVoting in a dental materials course using the methods described above proved overwhelmingly popular with students, but did not affect the outcome of a summative, end-of-course assessment based on comparison of exam scores in two consecutive classes of dental students. The benefits of using eVoting technologies, therefore, seem to be more in engaging the students and developing a stimulating and rewarding learning environment than in improving examination performance. The importance of a positive attitude towards learning, particularly in curriculum topics that are traditionally viewed as dull or dry by students, should not be underestimated. Because positive attitudes appear to be enhanced by active learner involvement in classroom lectures, electronic aids to teaching and learning such as eVoting are likely to continue to find application in dental education.
|
Acknowledgments
|
|---|
The author is indebted to Mr. Martin van Eker of the Centre for Medical Education, University of Bristol, for assistance with the ARS.
|
Author Information
|
|---|
Dr. Barbour is Lecturer in Biomaterials and Unit Organiser for Dental Biomaterials, Department of Oral and Dental Science, University of Bristol. Direct correspondence and requests for reprints to her at University of Bristol, Department of Oral and Dental Science, Lower Maudlin Street, Bristol BS1 2LY, UK; 44-117-9284184 phone; 44-117-9284780 fax; m.e.barbour{at}bristol.ac.uk.
|
References
|
|---|
- Hake RR. Interactive-engagement versus traditional methods: a six-thousand-student survey of mechanics test data for introductory physics courses. Am J Physics 1998; 66(1):64–74.
- Ramsden P. Teaching strategies for effective learning. In: Learning to teach in higher education. London: Routledge Falmer, 2003:145–75.
- Light G, Cox R. Lecturing: large group teaching. In: Learning and teaching in higher education: the reflective professional. London: Paul Chapman Publishing, 2001:97–114.
- Divaris K, Barlow PJ, Chendea SA, Cheong WS, Dounis A, Dragan IF, et al. The academic environment: the students’ perspective. Eur J Dent Educ 2008; 12(Suppl 1):120–30.[Medline]
- Caldwell JE. Clickers in the large classroom: current research and best-practice tips. CBE Life Sci Educ 2007; 6:9–20.[Abstract/Free Full Text]
- Turpin DL. Enhance learning with an audience response system. Am J Orthod Dentofacial Orthop 2003; 124:607.[Medline]
- Palmer EJ, Devitt PG, De Young NJ, Morris D. Assessment of an electronic voting system within the tutorial setting: a randomised controlled trial. Biomed Central Med Educ 2005; 5(1):24.
- Duggan PM, Palmer E, Devitt P. Electronic voting to encourage interactive lectures: a randomised trial. Biomed Central Med Educ 2007; 7:25.
- Stein PS, Challman SD, Brueckner JK. Using audience response technology for pretest reviews in an undergraduate nursing course. J Nurs Educ 2006; 45(11):469–73.[Medline]
- Moredich C, Moore E. Engaging students through the use of classroom response systems. Nurse Educator 2007; 32(3):113–6.[Medline]
- Johnson JT. Creating learner-centered classrooms: use of an audience response system in pediatric dentistry education. J Dent Educ 2005; 69(3):378–81.[Abstract/Free Full Text]
- Holmes RG, Blalock JS, Parker MH, Haywood VB. Student accuracy and evaluation of a computer-based audience response system. J Dent Educ 2006; 70(12):1355–61.[Abstract/Free Full Text]
- Draper SW, Brown MI. Increasing interactivity in lectures using an electronic voting system. J Comput Assist Learn 2004; 20:81–94.
- Knight JK, Wood WB. Teaching more by lecturing less. Cell Biol Educ 2005; 4:298–310.[Medline]
- Gulpinar MA, Yegen BC. Interactive lecturing for meaningful learning in large groups. Med Teacher 2005; 27(7):590–4.
- Costa ML, van Rensburg L, Rushton N. Does teaching style matter? A randomised trial of group discussion versus lectures in orthopaedic undergraduate teaching. Med Educ 2007; 41(2):214–7.[Medline]
- Barber M, Njus D. Clicker evolution: seeking intelligent design. CBE Life Sci Educ 2007; 6:1–20.[Abstract/Free Full Text]
Top
Abstract
Materials and Methods
