J Dent Educ. 71(9): 1160-1170 2007
© 2007 American Dental Education Association
Educational Methodologies |
Investigating the Relationship Between PBL Process Grades and Content Acquisition Performance in a PBL Dental Program
HsingChi von Bergmann, Ph.D.;
Kirsten R. Dalrymple, Ph.D.;
Shirley Wong, D.D.S.;
Charles F. Shuler, D.M.D., Ph.D.
Key words: problem-based learning, PBL process skills, content acquisition, assessment
Submitted for publication 01/04/07;
accepted 05/23/07
 |
Abstract
|
|---|
In this study, we examined the relationship between students’ problem-based learning (PBL) skills and their content acquisition as measured by traditional examinations. We conducted this investigation to evaluate the assumption that students’ learning in the small group setting of PBL as evaluated by their faculty facilitators was an accurate indicator of students’ learning as measured by problem analysis tests and traditional content acquisition tests. Parallel model reliability analyses were conducted to determine reliability for each year’s assessment components, which included multiple choice examinations, image-based computer tests, facilitators’ evaluations of students’ performance in the PBL small groups, and assessments that measured the students’ problem analysis and problem-solving skills. We also performed correlation tests to analyze the data. The reliability tests show that all assessment measures were consistently significant. There were predominantly significant correlations between process type assessment measures and the more traditional objective tests. When analyzed on a yearly basis, all of the correlations were significant. When analyzed on a trimester basis, all of the correlations were positive, with many being significant. The finding that the process grade revealed significant correlation with the other two assessment tools indicates that although process type evaluations may seem to be primarily subjective, they are an important metric for monitoring student progress.
The publication Dental Education at the Crossroads: Challenges and Change1 challenged dental educators to design curricula that prepare future dentists to think critically, to make evidence-based decisions, to behave empathically, to develop interpersonal communication skills, and to readily update their knowledge and clinical skills. The pedagogical approach chosen to introduce the curricular content represents an important component in the overall effort to achieve these curricular goals. Problem-based learning as a pedagogical approach has been incorporated into medical school curricula since the 1960s to enhance the clinical application of basic biomedical content and develop skills in critical thinking and inquiry-based learning.2 Many different approaches to the implementation of PBL have been developed in different medical schools, though each style of PBL shares common elements of small group learning and a focus on student inquiry.3 Incorporation of inquiry-based pedagogies like PBL into dental school curricula, however, has been slow.
A pilot program using PBL4 was introduced into the School of Dentistry at the University of Southern California (USCSD) in 1995; it incorporated principles closely aligned to those described by Bransford et al.5 After a six-year experience with the Pilot Parallel Track PBL program, the PBL format was adopted in fall 2001 for the entire class of 144 students.
To achieve the intended objectives associated with a substantive change in learning methods as outlined by Dental Education at the Crossroads1 requires fundamental changes to the assessment system of a program. The assessment approach should provide alignment with the new learning culture to ensure that the measurement of students’ learning is valid and beneficial to their education. Additionally, an educational environment that is student-centered, as Bransford et al.5 describe, needs to have mechanisms that give students timely and effective feedback on learning. The assessment system of USCSD was conceptualized to provide this alignment and to contain measures of both students’ generic learning process skills, also known at USCSD as PBL Process Skills, and measures of content acquisition. The use of these assessment measures is intended to help students gain insights into their own learning formatively (e.g., "How did I do in this case?") and summatively (e.g., "How did I do in this trimester?") with the overarching aim of developing the aforementioned professional competencies. As such, the generic learning process skills in the context of PBL at USCSD include skills related to critical reasoning, problem-solving, communication and feedback, and self-assessment.
The assessment system employed at this dental school is not unique; however, little research exists to inform educators if such a system is supported by achievement data. Two assumptions form the basis of this assessment system: 1) assessment by the faculty facilitator helps students improve their generic learning process skills, which are essential for individual and small group learning; and 2) students’ generic learning process skills in small groups influence their content acquisition as assessed by the content examinations. In this study, we examined the relationship between students’ PBL process skills as measured by the faculty facilitators and their content acquisition as measured by traditional-style examinations. We also studied whether student performance on two process-based examinations related to student performance on content acquisition examinations. These analyses are to test the assumption underlying the claim by Butler et al.6 that "true PBL is not skills-oriented—it is understanding-oriented."
|
Assessment Studies of PBL Programs
|
|---|
Essential elements in a PBL program include a student-centered focus, cooperative learning in small groups, regular formative assessment, and emphasis on student reflection on the thinking process. One recent study conducted by Nieminen et al.7 confirmed the significant relationship among group skills, case quality, and study outcomes. Taking first-year medical and dental students as the study subjects, these researchers found that group functioning and case quality were strongly correlated with students’ grades in a course examination. Furthermore, students’ perceptions of group functioning, case quality, and the quality of their own contribution were linked strongly with each other. The findings suggest that the elements inherent in a PBL curriculum combined to achieve the intended educational objectives. Nieminen et al.’s findings illuminate the arguments articulated by Dolmans et al.: PBL provides a situated, student-relevant, cooperative, and contextual pedagogical approach.8
Additional studies on assessment have also attempted to understand how specific elements in the PBL tutorial sessions related to aspects of students’ learning. Kaufman and Hansell9 and Whitfield and Xie10 both examined the relationship between facilitator ratings and student performance on examinations by investigating the ability of the facilitator to rate the extent of student content acquisition. The conclusions of both studies were that the judgment made by the facilitator of student knowledge was not consistent with student performance on examinations. These authors suggested that the facilitator ratings were inflated compared to the written exam results.
In our study, instead of focusing on measuring the facilitator’s perception of student content acquisition, we explored whether the facilitator’s rating of students’ generic learning process skills, essentially their cognitive and behavioral skills, in the tutorial setting were related to students’ content acquisition as measured by the objective assessment methods used in the PBL dental program. The program design is built on the belief that both cognitive skills (ability to analyze problems) and behavioral skills (working effectively in student groups through teamwork and effective communication) are critical in a PBL program. The assumption is that both sets of skills can be developed in a PBL environment.
Measurements in various educational settings have indirectly assessed the assumptions that 1) the development of cooperative learning skills is correlated with improved academic achievement,11–14 and 2) the ability to activate and elaborate upon prior knowledge is enhanced in a group learning setting and supports building of new knowledge frameworks.5 In testing these assumptions, investigators have found that structured and cooperative learning enhanced group members’ self-efficacy, motivation to learn, and perception of social support and diminished their test anxiety.12,15–17
Thus, when measuring both cognitive and behavioral skills, as observed by a PBL facilitator, we assumed that participation in small group discussions related to a specific patient problem can act as a catalyst in the activation of prior knowledge and stimulate interest in the subject matter. The conceptual framework for this study draws upon empirical studies of cognitive skills, behavioral skills, case quality, and cooperative learning to approach the issues surrounding assessment in PBL. We tested our assumptions using the PBL Process Skills Evaluation Form (see Figure 1
)18 administered by case facilitators to explore how measurement of PBL process skills in the small group sessions is related to measurements of content acquisition as evaluated by standardized tests19 and how these two aforementioned measurements related to performance on two process-based examinations.
|
Assessment System of the PBL Dental Program
|
|---|
It is commonly agreed among educators that assessment tools can track students’ progression of learning and serve as strong motivators for student achievement. Traditionally, assessment systems in dental education programs have included multiple choice question examinations, written examinations, oral examinations, and practical or performance-based examinations. In PBL-based educational programs, assessment of student performance by the facilitator during tutorial sessions is also employed to evaluate the process of learning.
The USCSD PBL dental program primarily follows the McMaster model utilized in medical education.20 The PBL dental students are given a case with several parts. For each part of the case, the student group analyzes the case via the process described in Figure 2.
In this model of PBL dental education, student performance is assessed by the following measurements: 1) PBL process grades provided by the group facilitator; 2) the two content acquisition examinations: the multiple choice question examination and the image-based, computer-based objective test; and 3) a problem analysis exercise (the triple jump) and a problem-solving exercise.
|
Assessment of PBL Process Skills
|
|---|
The facilitators use the PBL process skills assessment criteria (see Figure 1
) to assess students’ PBL process skills. Students also receive a copy of the PBL process skills assessment criteria at the beginning of their program. During each week of the case tutorial, based on the criteria, students receive a "below expected," "expected," or "above expected" mark for each of the four skills categories: group skills, learning skills, reasoning skills, and feedback skills. The group skills category evaluates if students actively participate in their group in a manner that promotes group function and learning. Learning skills is a category that measures student ability to identify, articulate, and evaluate learning topics relevant to the case and ideas formulated by the group as well as the ability to find and use appropriate resources to address the learning topics. The reasoning skills category evaluates whether students can summarize, evaluate, synthesize, apply relevant information to the case, and make connections between various concepts and contexts. The category of feedback skills assesses student ability to provide constructive criticism to individuals and to the group to advance learning in a case (for detailed aspects related to each category, please consult Figure 1). The assessment of PBL process skills by the facilitator results in a PBL process grade (PG) for each student. Facilitators provide feedback and discuss the PG with students on an individual basis upon completion of each case.
|
Assessment of Content Acquisition
|
|---|
In the USCSD PBL dental program, student acquisition of content knowledge is measured twice a trimester via two assessment methods, the multiple-choice question (MCQ) examination and the image-based, computer-based objective test (COMBOT). Both the MCQ and COMBOT are aligned with the major learning outcomes of the cases. These two examinations assess students’ acquisition of the content embedded in analyzed cases. Many test items for the MCQ examination are similar to the test items on the National Board Dental Examination Part I. The COMBOT differs from the MCQ in two respects. First, the COMBOT is administered via a computer, and second, each COMBOT question is composed of an image associated with an MCQ. The image is requisite for answering the MCQ. As an example, a case featuring topics related to bacterial infections might include an image of a gram stain asking students to identify a type of bacteria from a list of possibilities, based on morphology and staining patterns. Similar to the standardized tests in traditional lecture-based dental schools, there is more limited assessment of higher order cognitive skills, such as reasoning application of knowledge in the MCQ and COMBOT examinations.
|
Assessment of Problem Analysis and Problem-Solving Skills
|
|---|
During year one of the program, a student’s ability to analyze, gather information, and apply knowledge to a "problem" or patient case is measured twice a trimester using the triple jump (T3) examination.21 The goal of the T3 exercise is to assess the individual student’s ability to advance his or her understanding of a patient problem utilizing PBL process procedures that were modeled and reinforced in the small group learning environment. Each student obtains a unique problem case and is given a fixed amount of time to independently analyze the case utilizing the PBL process. During the initial analysis, the student identifies relevant facts, generates ideas relevant to the facts, and identifies topics to research. During the second "jump" of the exercise, the student researches the topics and applies that information back to the case and their original ideas. The final "jump" of the exercise is for the students to present their approach to analyzing the problem and their current understanding of it to a pair of faculty assessors. The final score for the student is an average of the two members’ assessment results.
In the second year, a problem-solving exercise, the P3, is used. While solving the problem is not the objective in the T3, it is the primary objective in the P3. For the P3, each student is given a case to analyze and a specified amount of time to conduct research. The student is expected to arrive at a differential diagnosis that can be defended based on the facts provided in the case. The P3 is a written exercise and is marked by expert faculty.
|
Methods
|
|---|
There were two null hypotheses for the study:
- H01: There is no relationship between the learning process skills measured by the case facilitators and learning process skills measured by either the T3 or the P3.
- H02: There is no relationship between the learning process skills measured by the assessment approaches conducted in the PBL dental program and the content acquisition measured using objective content-based examinations.
The Class of 2005 (144 students) admitted in fall 2001 was the first class to enroll in the USCSD after the transition of the entire dental school curriculum to a PBL program. The data used in this study were gathered during the first and second years of these students’ program of study. Among the 144 students, only 118 students’ assessment data were complete and thus utilized in this study.
While each PBL case is multidisciplinary in nature, there is a primary basic biomedical science or clinical dentistry thematic emphasis. Each trimester, students investigate six to eight cases. The PBL process grades (PGs) collected from each facilitator’s weekly assessment are calculated as an average score for each student for each case. Thus, by the end of each trimester, each student will have between six and eight PGs. Each group is facilitated by one faculty member. Facilitators are not assigned to facilitate the same student group for two consecutive cases both for logistical reasons and to provide student groups with exposure to multiple faculty members. Ninety-eight facilitators and thirty-three cases were included in this study. The results from twelve MCQs and twelve COMBOTs that were completed during the first two years by the Class of 2005 were available for analysis. Additionally, five T3 and five P3 examination results, obtained in the first two years of the Class of 2005’s program of study, were used in this study.
Using the computer program Statistical Package for the Social Sciences (SPSS),22 the data were subjected to parallel model reliability analyses and correlation tests. The four components of the PG grades were equally weighted, as were the PG grades obtained from each case.
Parallel model reliability analyses were conducted to determine goodness-of-fit for each year’s MCQ, COMBOT, PG, T3, and P3. This method used maximum likelihood to test if all items have equal variances and error variances. Cronbach’s alpha is the maximum likelihood estimate of the reliability coefficient when the parallel model is assumed to be true.22 If the chi-square goodness-of-fit significance for the parallel model is smaller than or equal to .05, the researcher rejects the null hypothesis that the items have equal variances in the population. The parallel model was used because all the assessment measures are designed to measure a certain type of student learning. These measures are administered independently from one another. It is assumed that some may fail to measure what is intended to measure, but only when all fail its task will the assessment system fail. Furthermore, because the measurement precision (related closely to the validity of measurement design or implementation) is not uniform across the scale of measurement, the reliability tests, such as alpha, typically applied in classical testing theory were not suitable for our study.
To determine if relationships exist within the PBL dental program’s assessment system, we also performed correlation tests to analyze the data at a two-tailed level with p-value equals to .05 as the indicator of significance. Specifically, we tested relationships between the PG and T3, PG and P3, PG and MCQ, PG and COMBOT, T3 and COMBOT, T3 and MCQ, P3 and COMBOT, and P3 and MCQ.
|
Results
|
|---|
Table 1 is a summary of the reliability coefficients for the assessment measures—MCQ, COM-BOT, PG, T3, and P3 in Year 1 and Year 2. The reliability tests show that all assessment measures were significantly consistent (reliability values ranged from .36 to .92); consistency measures (icc: measures of consistency or agreement of values within cases) of all assessment measures were significant at p<.01 level. The P3 measure has the lowest reliability value (.36), whereas the Year 1 MCQ measure has the highest reliability value (.92). MCQ, COMBOT, and PG had a higher reliability value in Year 1 than in Year 2. The two standardized examinations, MCQ and COMBOT, had a higher reliability value than the process skills assessment measures PG, T3, and P3. The process skills measured by PBL facilitators (PG) had a higher reliability value than the process skills measured by T3 and P3.
Our two null hypotheses stated that 1) there is no relationship between the process skills measured by case facilitators and measured by either the T3 or P3, and 2) there is no relationship between the process skills measures and the content acquisition measures. As shown in Table 2, we found that there were statistically significant correlations (ranging from .20 to .45) for all assessment measures, indicating that students who were recognized as having good process skills in tutorial sessions also did well in their T3s and content acquisition tests. Both null hypotheses were rejected.
View this table:
[in this window]
[in a new window]
|
Table 2. Correlation coefficients for the MCQ, COMBOT, PG, T3, and P3 of the PBL dental program class of 2005 in years 1 and 2
|
|
The relationships were stronger in Year 1 than in Year 2 for the process skills measures (PG, T3, and P3) and the content acquisition measures (MCQ and COMBOT). A significantly stronger relationship existed between PG and T3 in Year 1 than the relationship between PG and P3 in Year 2.
Based on these results, we found that a significant relationship exists between the measures of process skills and the measures of content acquisition in the PBL dental program. Subsequently, we attempted to understand these relationships at the trimester level, case by case, and basic biomedical science cases versus cases with a clinical dentistry emphasis.
Table 3 shows the correlation test results between process skill measures and content acquisition measures trimester by trimester. Significant positive relationships are found in all three trimesters in Year 1 between the PG and MCQ and between the PG and COMBOT. In trimesters 1 and 2, the T3 also has a significant correlation with PG, MCQ, and COMBOT. In the third trimester, the relationship between the T3 and other assessment measures is not significantly correlated. In Year 2, the relationships between PG and MCQ or COMBOT are not significant across all three trimesters as they are in Year 1. The relationships among P3 and MCQ, COMBOT, and PG were not as strong as compared to T3 with those measures.
View this table:
[in this window]
[in a new window]
|
Table 3. Correlation coefficients for the MCQ, COMBOT, PG, T3, and P3 of the PBL dental program class of 2005 in years 1 and 2
|
|
In Table 4 and Table 5, the correlation test results between process skill measures and content acquisition measures for basic sciences cases versus clinical dentistry cases in Year 1 and Year 2 are presented. Forty percent of the correlation tests were significant in Year 1 for the cases with a basic biomedical science theme. Thirty percent of the correlation tests were significant in Year 1 for the cases with a major theme addressing a clinical dentistry area.
Compared to Year 1, there were smaller differences in the number of significant relationships between the basic biomedical science cases and clinical dentistry cases in Year 2 (12.5 percent for basic science cases; 14 percent for clinical dentistry cases).
The correlation test results of the relationship between process skill measures and content acquisition measures case by case over the two years are presented in Table 6. For this analysis, the data were used to test "case quality," inspired by the findings reported by Nieminen et al.7 Their study suggested that group function skills, case quality, and study outcomes are significantly related. Using our data, we investigated the case quality, which is a construct defined by relationships between PG, a group function indicator, and study achievement measures such as T3, P3, MCQ, or COMBOT.
Table 6 displays the same patterns as the previous three tables: there are more significant relationships between the process skill measures and the measures of content acquisition in Year 1 than there are in Year 2 at the individual case level. The case quality is reflected by how strongly related the process skills measures are to the content acquisition assessment measures. Thus, according to the results in Table 6, thirteen out of seventeen cases (76 percent) in Year 1 were considered "quality cases," whereas in Year 2, six out of sixteen (38 percent) were "quality cases." An important note is that the facilitators vary with each case, and as the program expanded, greater numbers of facilitators were involved, contributing an element to the variation.
|
Discussion
|
|---|
We initiated this analysis of data to evaluate our assumption that students’ PBL process skills in the small group setting as evaluated by their facilitators were accurate indicators of students’ learning as measured by problem analysis tests (T3, P3) and content acquisition tests (MCQ, COMBOT). Our data analyses show that there are statistically significant relationships between the measures of PBL process grades given by facilitators for performance in the small group sessions and the T3, between the measures of PBL process grades and the MCQ or COMBOT, and between the T3 and the MCQ or COMBOT. These relationships provide evidence that the facilitator’s assessment of students based on their performance in the small group learning environment is a reliable indicator of performance on content acquisition examinations. We did not perform regression analyses in our study to find the predictability values of PBL process grades toward content acquisition measures. One key reason is that the correlation coefficients, although significant, are not large enough across the board. Furthermore, the reliability tests indicating the consistency within PBL process skills measures may also not be strong enough, for the values are relatively smaller compared to the objective measures.
In spite of these limitations, there is sufficient evidence to conclude that relationships exist between the PBL process skill measures and content acquisition measures. This finding provides PBL educators with critical support for establishing an assessment system that is consistent with the program principles: PBL process skills as reinforced by the case facilitator make a difference in students’ achievement.
Dolmans et al.8 argued that what is currently needed in PBL research is to "contribute towards a better understanding of why and how these theories or concepts work or not in PBL and under what circumstances" (p. 737). In the USCSD PBL dental program, we found that when students learn in a small group, a theory-based group skills assessment7 needs to be constructed for measurement and reinforcement of desired PBL process skills (see Figure 1). Our data analyses show that, in general, the PBL process skills assessed by the facilitator in the small group setting do relate to students’ summative learning achievement.
The central measure of this study is the PBL process grade (PG) that is determined by the faculty facilitators working with individual student PBL groups. Thus, one critical contextual variable is the qualification of the PBL facilitators. During the PBL implementation in 2001 in trimesters 1 and 2, the students had the greatest number of experienced facilitators as compared with the other trimesters. "Experienced" refers to previous PBL facilitation experience, graduates of the pilot PBL program, and/or PBL professional development training. Thus, the experienced facilitators were most familiar with the goals of PBL and small group learning essentials as compared to less experienced facilitators in the later trimesters.8 The PGxMCQ and PGxCOMBOT correlation coefficients for the first trimester were highest when compared to all other trimesters.
Evaluation of the PG descriptive statistics found that the PG in the first trimester was lowest compared to the other five trimesters. We also found that the MCQ and COMBOT scores remained relatively consistent throughout all six trimesters. As students progressed in the program, the facilitator ratings of student performance during case sessions gradually improved coincident with student experience with the pedagogy, but a similar increase was not reflected in students’ performance on MCQ nor COMBOT. This observation revealed that students in the first trimester were less experienced and their experienced facilitators were more rigorous in their assessment of students’ PBL process skills. Student performance in case sessions improves as the students gain more experience and reflect on their own performance based on facilitator assessment feedback. Consequently, student performance in the group learning environment will become more consistent and thus less sensitive to variance that continues to occur in the MCQ and COMBOT. PG assessment is thus based on a more consistent element of student performance than the MCQ and COMBOT, which measure new content introduced in the cases that were investigated. Since the PG becomes a consistent strength of the students, the lack of appreciable variance is a major contributor to drop in the instances of significant relationship in Year 2.
Ratings of student group process skills by facilitators have been criticized in some studies as being an unreliable measure of student performance.19,20 The psychometric weakness has, in part, been attributed to facilitators who demonstrate a reluctance to give poor evaluations despite having evidence of poor student performance.23–25 This situation has been proposed to exist both for new facilitators, who may be unfamiliar with applying learning skills assessment criteria, and experienced facilitators, who become more familiar with a cohort of students, a phenomenon akin to the "halo effect."26 In Year 2 of the program, in particular, fewer experienced facilitators were available to work with the PBL student groups. During this time it was generally observed that the less experienced facilitators assessed students much more frequently in the above expected category, leading to less variation in student assessment measures. In such circumstances, the process grade tends to lose sensitivity in detecting the variance that continued to occur in the MCQ or COMBOT examinations.
The results of this study have two critical outcomes for PBL educators: 1) for effective cultivation of group process skills in a PBL environment, it is critical to have a theory-based assessment measure to assist PBL facilitators; and 2) the effectiveness of the PBL facilitator is critical to reaching the overall objective of PBL in student-centered, small group, and case-based learning. The findings reported here indicate that cultivation of students’ learning skills by experienced PBL facilitators makes a difference in the students’ overall performance in all aspects of the curriculum.
Evaluation of the pair-wise comparisons PGxT3, T3xMCQ, and T3xCOMBOT during Year 1 provided evidence of more significantly positive correlations between the T3 and content acquisition tests when compared to the correlations between the T3 and PBL process grades. Initially, this may seem counterintuitive since both the PG and T3 examination are intended to assess students’ PBL process skills. However, our results may reflect the difference between the PG and T3 assessments. Since the T3 is an individual assessment, only two of the four PBL process skills, reasoning and learning, are assessed, while in the small group learning format, a student is assessed with respect to all four PBL process skills (Group, Learning, Reasoning, Feedback). Valle et al.27 found that reasoning skill and independent studying (a similar construct to our "learning skills" category of the process skills) are correlated. Because there are two overlapping process skills between T3 and PG, we identified significant relationships between these two measures, which supports Valle et al.’s finding.
|
Conclusion
|
|---|
The results of our study show that there were significant correlations between process type assessment measures and the more traditional content acquisition tests. When analyzed on a yearly basis, all of the correlations were significantly positive. When analyzed on a trimester basis, all of the correlations were positive, with many being significant. The finding that the process grade revealed a significant correlation with the other two assessment tools indicates that although process type evaluations completed by the faculty who facilitate small group, case-based learning may seem to be relatively subjective, they are an important metric for monitoring student progress in PBL curricula and potentially in other educational formats that incorporate types of small group learning.
|
Footnotes
|
|---|
Dr. von Bergmann is Associate Professor, Faculty of Education, University of Calgary; Dr. Dalrymple is Assistant Professor, Division of Diagnostic Sciences, School of Dentistry, University of Southern California; Dr. Wong is Assistant Professor, Clinical Dentistry, School of Dentistry, University of Southern California; and Dr. Shuler is Dean of the Faculty of Dentistry, University of British Columbia. Direct correspondence and requests for reprints to Dr. HsingChi von Bergmann, EDT 726, Faculty of Education, University of Calgary, Alberta, Canada T2N1N4; 403-220-6160 phone; 403-282-8479 fax; hsingchi{at}ucalgary.ca.
|
REFERENCES
|
|---|
- Institute of Medicine. Dental education at the crossroads: challenges and change. Washington, DC: National Academy Press, 1995.
- Barrows HS, Tamblyn R. Problem-based learning: an approach to medical education. New York: Springer Publishing Company, 1980.
- Maudsley G. Do we all mean the same thing by "problem-based learning"? A review of concepts and formulation of the ground rules. Acad Med 1999; 74:178–85.[Medline]
- Barrows HS. The essentials of problem-based learning. J Dent Educ 1998; 62(9):630–3.[Medline]
- Bransford JD, Brown AL, Cocking RR. How people learn: brain, mind, experience, and school. Washington, DC: National Academy Press, 1999.
- Butler R, Inman D, Lobb D. Problem-based learning and the medical school: another case of the emperor’s new clothes? Adv Physiol Educ 2005; 29:194–6.[Abstract/Free Full Text]
- Nieminen J, Sauri P, Lonka K. On the relationship between group functioning and study success in problem-based learning. Med Educ 2006; 40:64–71.[Medline]
- Dolmans DHJM, De Grave W, Wolfhagen IHAP, van der Vleuten CPM. Problem-based learning: future challenges for educational practice and research. Med Educ 2005; 39(7):732–41.[Medline]
- Kaufman D, Hansell M. Can non-expert PBL tutors predict their students’ achievement: an exploratory study. Acad Med 1997; 72:S16–S18.[Medline]
- Whitfield CF, Xie SX. Correlation of problem-based learning facilitator’s scores with student performance on written exams. Adv Health Sci Educ 2002; 7:41–51.
- Hoek D, van den Eeden P, Terwel J. The effects of integrated social and cognitive strategy instruction on the mathematics achievement in secondary education. Learning and Instruction 1999; 9(5):427–48.
- Ghaith GM. The relationship between cooperative learning, perception of social support, and academic achievement. System 2002; 30(3):263–73.
- Shachar H, Fischer S. Cooperative learning and the achievement of motivation and perceptions of students in 11th grade chemistry classes. Learning and Instruction 2004; 14(1):69–87.
- David TJ, Patel L. Adult learning theory, problem-based learning and paediatrics. Arch Dis Child 1995; 73(4): 357–63.[Medline]
- Griffin MM, Griffin BW. An investigation of the effects of reciprocal peer tutoring on achievement, self-efficacy, and test anxiety. Contemp Educational Psychol 1998; 23(3):298–311.
- Klein JD, Schnackenberg HL. Effects of informal cooperative learning and the affiliation motive on achievement, attitude, and student interactions. Contemp Educational Psychol 2000; 25(3):332–41.
- Gillies RM. The behaviors, interactions, and perceptions of junior high school students during small group learning. J Educational Psychol 2003; 95(1):137–47.
- Mennin S. Faculty and student guide to problem-based learning tutorials in phase I curriculum. Albuquerque: Office of Teacher and Educational Development, University of New Mexico School of Medicine, 2002.
- Messick S. Validity. 3rd ed. New York: Macmillan, 1989.
- Fincham AG, Shuler CF. The changing face of dental education: the impact of PBL. J Dent Educ 2001; 65(6):406–21.[Abstract]
- Smith RM. The triple-jump examination as an assessment tool in the problem-based medical curriculum at the University of Hawaii. Acad Med 1993; 68:366–72.[Medline]
- SPSS-X user’s guide. 3rd ed. Chicago: SPSS Inc., 1988.
- Cunnington J. Evolution of student assessment in McMaster University’s MD programme. Med Teacher 2002; 24(3):254–60.
- Eva KW. Assessing tutorial-based assessment. Adv Health Sci Educ Theory Practice 2001; 6(3):243–57.
- Des Marchais JE, Vu NV. Developing and evaluating the student assessment system in the preclinical problem-based curriculum at Sherbrooke. Acad Med 1996; 71(3):274–83.[Medline]
- Cohen G. Do final grades reflect written qualitative evaluations of student performance? Teach Learn Med 1993; 5:10–5.[Medline]
- Valle R, Petra L, Martinez-Gonzaez A, Rojas-Ramirez JA, Morales-Lopez S, Pina-Garza B. Assessment of student performance in a problem-based learning tutorial session. Med Educ 1999; 33:818–22.[Medline]
Top
Abstract
Assessment studies of pbl...
