J Dent Educ. 71(6): 819-824 2007
© 2007 American Dental Education Association
Critical Issues in Dental Education |
Do Tutor Expertise and Experience Influence Student Performance in a Problem-Based Curriculum?
Sang E. Park, D.D.S., M.M.Sc.;
Srinivas M. Susarla, B.A.;
Cheri K. Cox, D.M.D.;
John Da Silva, D.D.S. M.P.H, Sc.M.;
T.H. Howell, D.D.S., M.M.Sc.
Key words: student performance, problem-based curriculum, PBL, tutor expertise
Submitted for publication 10/30/06;
accepted 02/16/07
 |
Abstract
|
|---|
The purpose of this study was to evaluate the association between student performance and tutor expertise and experience in the Restorative Treatment course in a problem-based learning (PBL) dental curriculum. This retrospective study evaluated a cohort of 206 third-year dental students at Harvard School of Dental Medicine who were enrolled in the Restorative Treatment course between 2000 and 2005. Tutor expertise in prosthodontics and prior tutoring experience were measured against student performance in five areas: 1) tutorial session, 2) midterm examination, 3) final examination, 4) preclinical laboratory, and 5) overall cumulative course grade. Student performance in each of the five areas measured against combinations of tutor subject matter expertise and tutoring experience level showed no statistically significant difference between the groups in the overall course grade. However, student performance in the expert group showed a significant difference in the final examination compared to the nonexpert group (p<0.05). Also, students in the group consisting of expert tutors with no experience performed better on the final examination compared to those students who had expert tutors with experience (p<0.05). The results indicate that overall student performance in the problem-based Restorative Treatment course is not affected by tutor expertise or prior tutoring experience.
Problem-based learning (PBL) is a case-based, self-directed learning method designed to develop critical thinking and problem-solving skills, including the acquisition, integration, and application of new knowledge.1 The educational characteristics and objectives of PBL include the development of effective student-centered, self-directed learning skills and the construction of knowledge through an integrated reasoning process for application in clinical cases.2,3 The problem-based learning curriculum has been widely applied in medical education since its introduction in 1965 at McMaster University in Canada.4,5 Although its role in dental education is limited, an increasing number of institutions in recent years have integrated PBL into their educational programs.
Harvard School of Dental Medicine (HSDM) introduced the PBL model into its four-year D.M.D. curriculum in 1994 using a hybrid PBL methodology for basic science, preclinical, and clinical dental education.6–8 The HSDM curriculum is considered to be a hybrid because small-group, case-based learning in tutorials guided by faculty facilitators is combined with the traditional lecture-format curriculum. The PBL curriculum was implemented in an effort to encourage critical thinking, scientific inquiry, and proactive student-directed learning within a case-based curriculum.
Recent literature supports the hypothesis that PBL offers advantages to traditional dental education.9–11 Implementation of the PBL model into the dental curriculum showed significantly improved results at standardized examination and student performance levels. Higher performance on Part I of the National Board Dental Examination (NBDE) was observed in two independent studies at HSDM and the University of Southern California after adoption of the PBL curriculum.9,10 Another study at the University of British Columbia showed that students in a hybrid-PBL course enjoyed the learning process and were rated higher in treatment planning.11
In a PBL curriculum, the role of the faculty member is very different from the traditional teacher’s role in a conventional teaching format. Tutors of small-group sessions, or tutorials, are designed to function as facilitators of discussion and guide self-directed student learning.12 The role of the tutor is not to convey knowledge and teach as a content expert, but to encourage student-to-student interaction and facilitate the learning process by working with the group to explore the problem.12,13
Due to the facilitative role of the tutor, the level of tutor content expertise, theoretically, should not affect student performance in the PBL model. Howard Barrows, a leader in the field of PBL medical education, suggests that facilitation skill is more important than subject matter or content expertise.13 However, the best tutor would be someone who possesses relevant subject matter knowledge and is a good facilitator.13 Some studies suggest that content expertise can have a negative effect on tutoring by leading to less student-to-student discussion.14,15 Observations made on the academic achievement of students when compared by the different levels of tutor subject matter expertise are largely inconclusive. Schmidt et al.16 showed that students performed better and spent more time on self-directed study when guided by experts. Similarly, Davis et al.17 found that student test performance was increased when tutors had experiences in research or advanced disciplinary training in the tutorial subject matter. Swanson et al.,18 however, found no difference in levels of student performance based on tutor expertise.
The literature is replete with efforts to measure the effects of tutor expertise on student performance in the PBL model in medical education, but this topic has not been given the same attention in dental education. Several studies in dental literature have examined the role of tutor subject matter expertise in the PBL model; however, there have not been studies conducted measuring tutor experience levels against student performance in dental education specifically. In our study, the qualification of tutor expertise and prior tutoring experience was defined and measured against student performance.
The aim of this study was to measure whether tutor expertise and prior tutoring experience influence student performance in the Restorative Treatment course within a PBL dental curriculum at the Harvard School of Dental Medicine. The hypothesis of the study was that tutor expertise and experience level do not influence student performance in the PBL curriculum.
|
Materials and Methods
|
|---|
This study was a retrospective study of 206 third-year dental students enrolled at HSDM between 2000 and 2005. Each year students were randomly assigned to tutorial groups ranging in size from seven to nine students per group for the problem-based teaching in the Restorative Treatment course, which was the focus of the study. The tutors were also assigned randomly to one of the four tutorial student groups. The Restorative Treatment course profiled in this study was a six-month block, and the students remained with the same tutor throughout the entire block. This course provides students with the didactic knowledge and preclinical skill development to formulate a comprehensive treatment plan for the restorative needs of the patient. The treatment concepts of fixed and removable prosthodontics are introduced through discussion of clinical cases.
Expertise and prior tutoring experience were defined with the following criteria:
- Qualification of Expert: Content experts were tutors with specific background knowledge in the subject matter. In this study, the qualification of expert was given to a prosthodontics specialist in the field of dentistry.
- Qualification of Nonexpert: The qualification of nonexpert was reserved for general dentists.
- Prior Tutoring Experience: Tutors were divided based on whether or not they had been a PBL tutor for one year or more in the Restorative Treatment course.
Selected tutors attended faculty workshops to calibrate them to the facilitator role and were encouraged to observe another faculty member perform this role in a tutorial group.
Student performance in the PBL Restorative Treatment course was measured in relationship to tutor expertise and prior tutoring experience. The students’ grades for the tutorial session, preclinical laboratory, midterm examination, final examination, and the overall cumulative course grade were each measured against the various tutor expert and experience groups. The study was approved by the Institutional Review Board at Harvard Medical School (IRB protocol # M12815-101).
Data were obtained from the Office of the Registrar at the Harvard School of Dental Medicine and coded to ensure anonymity and confidentiality. These data were entered into a statistical database (SPSS v.11.0, © SPSS Inc., Chicago, IL) for analysis. Descriptive statistics were computed to compare the different study groups and identify any potential confounders and effect modifiers. Bivariate statistics were computed to compare the different study groups with regard to the outcome variables. Nonparametric methods were used for bivariate comparisons. For all analyses, a p-value <0.05 was considered statistically significant.
|
Results
|
|---|
Descriptive statistics for the study population, categorized according to the level of tutor expertise (expert versus non-expert) are summarized in Table 1
. There were no statistically significant differences in the average undergraduate grade point average (GPA), Dental Admission Test/Perceptual Ability Test (DAT/PAT) scores, and NBDE Part I scores between those students with expert tutors and those with nonexpert tutors (Table 2). In addition, there were no statistically significant differences in the proportions of students who achieved honors/pass/fail grades between those in the expert and nonexpert groups for the outcome measures, except the final examination.
Table 3 summarizes the association between tutor expertise and experience in terms of student distributions. Interestingly, all tutors with experience were classified as expert tutors, and concurrently, all nonexpert tutors had no prior experience. As such, a separate analysis was conducted to evaluate the association between tutor experience and student performance within the expert group. These comparisons are summarized in Table 4. For the outcome measures, there were no statistically significant differences between the study groups with regard to performance in tutorial sessions, preclinical laboratory exercises, midterm examinations, or overall course grades. There was a statistically significant difference between the groups for the final examination performance (p<0.05). Students with tutors who were experts without experience were statistically significantly more likely to receive a grade of honors on the final examination when compared to students who had tutors that were experts with experience as group facilitators.
There were no statistically significant differences in the average values for potentially confounding variables (undergraduate GPA, DAT/PAT, and NBDE Part I scores) as shown in Table 5. Additionally, the distributions of honors/pass/fail grades were statistically equivalent between those students with experienced tutors and those without experienced tutors.
|
Discussion
|
|---|
The results of our study revealed that no statistically significant difference exists in student performance in the overall course grade between students taught by expert tutors and nonexpert tutors after controlling for the effects of other covariates. There was also no statistically significant difference in student performance in the overall course grade between students taught by experienced tutors or tutors with no prior experience. The results support the model that the role of the tutor is as a facilitator and not a purveyor of knowledge. Final examination was the only outcome measure that showed a significant difference in relation to tutor expertise or experience level. This finding may have been attributed to the fact that the same experts were involved in the production of questions for this test.
This study revealed a lack of any uniform associations between tutor expertise and experience level and overall outcomes (student performance) in a restorative dentistry course. It is possible that the sample size evaluated may be insufficient to test the complex effects of various covariates on multiple outcomes. The relatively small variance in outcome measures seen in this cohort increases the difficulty of evaluating the incremental effects of the predictors on the outcome in the setting of a small sample size. In addition, unequal student distribution in groups of tutor expertise and experience levels contributed to the limitation on data interpretation.
Many studies have been reported comparing tutor subject matter expertise in comparison to student performance in PBL medical education. Matthes et al.19 conducted a study comparing medical student performance and motivation among groups of students led by expert and nonexpert tutors. The study sample was a group of 787 third-year medical students enrolled in a PBL course of basic pharmacology. They found that neither tutors’ subject matter expertise nor teaching method expertise showed any influence on their respective PBL groups’ mean test scores on a written exam. Although many studies exist from schools with problem-based curricula, the results are largely conflicting and inconsistent.14–18 Some studies found tutor expertise to be an advantage for learning outcome and student performance,16 while others found no difference in student performance between groups led by experts and those led by nonexpert tutors. De Grave et al.20 conducted a study using the Tutor Intervention Profile instrument to study the behaviors of different tutors. The Tutor Intervention Profile instrument includes four dimensions of tutor behavior: elaboration, directing the learning process, integration of knowledge, and stimulating interaction and individual accountability. This study found that students responded better to tutors who focused on the learning process in comparison to tutors who stressed content more; however, the difference was not found to be statistically significant. The results of this study showed that tutors who stressed content more than process were mostly expert tutors.
Inconclusive debate within the literature as to whether student performance is dependent on tutor expertise is related to less than stringent definition of expert versus nonexpert. Lack of established standards to determine expertise may be a factor contributing to inconsistent findings. Definition of content expertise varies from the level of training of the tutor to the tutor’s specific knowledge in the field of study. A study by Bochner et al.21 examined expert versus nonexpert tutors in a problem-based learning curriculum in dental education at Harvard School of Dental Medicine. The study compared how students evaluate expert tutors and nonexpert tutors and compared tutor evaluations with different standards of defining expertise. Three different classifications of expertise were used in this study, and student tutor evaluations were compared for each. Based on the literature, it is apparent that it is very important to define what constitutes an expert when measuring outcomes with respect to the PBL curriculum model for education. Categorization of expert versus nonexpert tutors in our study did not account for possible overlap in the knowledge and insight they brought to each tutoring session.
Previous exposure to problem-based learning may also contribute to the manner in which both students and tutors behave in a PBL dental curriculum. Students who have little or no experience with problem-based learning often rely more heavily on their tutors.17 The third-year dental students included in this study have had ample experience with PBL educational systems prior to participating in the Restorative Treatment course; therefore, the type of tutor assigned to the group could have had a minimal effect. Concurrently, tutors with previous exposure to PBL as students may have an advantage in understanding the role of the tutor as a facilitator, providing formative feedback only when necessary, and not functioning as a teacher or conveyor of knowledge. Even though they may have had no prior tutoring experience, these tutors may be more familiar with facilitating student-directed problem-solving and student-led discussions because of their experiences while in school. High variability among tutor personalities leading to a wide range of tutoring style, regardless of tutor expertise or prior tutoring experience, were not accounted for in our study and may have contributed to the results.
Motivation of the student as a primary determinant of achievement in PBL was identified by van Berkel and Schmidt.22 Commitment measured in terms of tutorial attendance was observed to be the key factor of achievement compared to tutor performance, small group function, level of prior knowledge, or interest in subject matter. Motivation and commitment level of both students and tutors are probably significant factors when assessing the educational benefits of a PBL curriculum and should be considered in future studies. To measure the impact of tutor expertise and prior tutoring experience on student performance, the aforementioned parameters must be considered for an accurate assessment.
Our study supports the problem-based learning model, suggesting that the task of the tutor in a PBL curriculum should be to facilitate student learning rather than to give students information. No significant difference was found in the overall performance between students who had tutors who were experts or nonexperts. The question remains whether an expert and experienced tutor is necessary for a successful PBL tutorial environment. This study supports the conclusion that a tutor need not be an experienced expert to achieve high student performance levels. Implementation of teaching by postgraduate residents in the subject matter may thus be a feasible approach to ease faculty shortages in dental schools nationwide.
|
Acknowledgments
|
|---|
The authors would like to acknowledge the invaluable assistance of Diane Spinell and Carole Chase.
|
Footnotes
|
|---|
Dr. Park is Senior Tutor, Office of Dental Education; Mr. Susarla is a D.M.D. candidate; Dr. Cox is a recent graduate; Dr. Da Silva is Director of Advanced Graduate Education, Office of Dental Education; and Dr. Howell is Dean for Dental Education and A. Lee Loomis Professor of Periodontology—all at the Harvard School of Dental Medicine. Direct correspondence and requests for reprints to Dr. Sang E. Park, Harvard School of Dental Medicine, 188 Longwood Avenue, Boston, MA 02115; 617-432-2374 phone; 617-432-3881 fax; sang_park{at}hsdm.harvard.edu.
This research was presented at the 2006 ADEA Annual Session, Orlando, FL.
|
REFERENCES
|
|---|
- Barrows HS. The essentials of problem-based learning. J Dent Educ 1998; 62(9):630–3.[Medline]
- Barrows HS. A taxonomy of problem-based learning methods. J Med Educ 1986; 20(6):481–6.
- Albanese MA, Mitchell S. Problem-based learning: a review of the literature on its outcomes and implementation issues. Acad Med 1993; 68(1):52–81.[Medline]
- Neufeld V, Barrows H. The "McMaster philosophy": an approach to medical education. J Med Educ 1974; 49(11):1040–50.[Medline]
- Barrows HS, Tamblyn RM. Problem-based learning: an approach to medical education. New York: Springer, 1980.
- Matlin KS, Libert E, McArdle PJ, Howell TH. Implementing the problem-based curriculum at Harvard School of Dental Medicine. J Dent Educ 1998; 62(9):693–708.[Medline]
- Howell TH, Matlin K. Damn the torpedoes—innovations for the future: the new curriculum at the Harvard School of Dental Medicine. J Dent Educ 1995; 59(9):893–8.[Medline]
- Moore GT. Initiating problem-based learning at Harvard Medical School. In: Boud D, Feletti G, eds. The challenge of problem-based learning. 2nd ed. London: Kogan Page, 1997:73–80.
- Susarla SM, Medina-Martinez N, Howell TH, Karimbux NY. Problem-based learning: effects on standard outcomes. J Dent Educ 2003; 67(9):1003–10.[Abstract]
- Shuler CF, Fincham AG. Comparative achievement on National Board Dental Examination Part I between dental students in problem-based learning and traditional educational tracks. J Dent Educ 1998; 62(9):666–70.[Medline]
- Walton JN, Clark DC, Glick N. An outcomes assessment of a hybrid-PBL course in treatment planning. J Dent Educ 1997; 61(4):361–7.[Abstract]
- Tipping J, Freeman RF, Rachlis AR. Using faculty and student perceptions of group dynamics to develop recommendations or PBL training. Acad Med 1995; 70(11):1050–2.[Medline]
- Barrows HS. The tutorial process. Springfield: Southern Illinois University, 1987.
- Wilkerson L, Hafler JP, Liu PA. Case study of student-directed discussion in four problem-based tutorial groups. In: Proceedings of the thirtieth annual conference on research in medical education. Acad Med 1991; 66(Suppl): S79–S81.[Medline]
- Silver M, Wilkerson LA. Effects of tutors with subject expertise on the problem-based tutorial process. Acad Med 1991; 66(5):298–300.[Medline]
- Schmidt HG, van der Arend A, Moust JH, Kokx I, Boon L. Influence of tutors’ subject matter expertise on student effort and achievement in problem-based learning. Acad Med 1993; 68(10):784–91.[Medline]
- Davis WK, Nairn R, Paine ME, Anderson RM, Oh MS. Effects of expert and non-expert facilitators on the small-group process and on student performance. Acad Med 1992; 67(7):470–4.[Medline]
- Swanson DB, Stalenhoef-Halling BF, van der Vleuten CPM. Effect of tutor characteristics on test performance of students in problem-based curriculum. In: Bender W, Hiemstra RJ, Scherpbier AJJA, Zwierstra RP, eds. Teaching and assessing clinical competence. Groningen, The Netherlands: Boek Werk Publications, 1990:129–34.
- Matthes J, Marxen B, Linke RM, Antepohl W, Coburger S, Christ H, et al. The influence of tutor qualification on the process and outcome of learning in a problem-based course of basic medical pharmacology. Naunyn Schmiedebergs Arch Pharmacol 2002; 366(1):58–63.[Medline]
- De Grave WS, Dolmans DH, van der Vleuten CP. Profiles of effective tutors in problem-based learning: scaffolding student learning. Med Educ 1999; 33(12):901–6.[Medline]
- Bochner D, Badovinac RL, Howell TH, Karimbux NY. Tutoring in a problem-based curriculum: expert versus nonexpert. J Dent Educ 2002; 66(11):1246–50.[Abstract]
- van Berkel HJM, Schmidt HG. Motivation to commit oneself as a determinant of achievement in problem-based learning. Higher Educ 2000; 40(2):231–42.
Top
Abstract
Materials and Methods
