| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
Critical Issues in Dental Education |
Key words: competency-based education, competency, clinical evaluation, curriculum, dental education
Submitted for publication 04/02/07; accepted 10/03/07
 |
Abstract |
|---|
 Top Abstract MethodsResultsDiscussionConclusionReferences |
|---|
The concept was introduced in dental education in 1993 with the publication of Chambers’s article "Toward a Competency-Based Curriculum"1 and a plenary session at the annual meeting of the American Dental Education Association (ADEA; then, American Association of Dental Schools, AADS) featuring an exploration of the potential impact of competency on various aspects of education in the profession.2–8 The definition proposed by Chambers at that time was the following: "Competencies are skills essential to beginning the practice of dentistry and allied dental practice. Competencies combine appropriate supporting knowledge and professional attitudes, and they are performed reliably in natural settings without assistance" (p. 791).1 The accreditation standards for dental education were changed in 1997 to require a competency-based approach, with the following definition: "Competent: The levels of knowledge, skills, and values required by the new graduate to begin independent, unsupervised dental practice" (p. 21).9
As demonstrated in Figure 1
, the number of publications on competency-based dental education appearing in the Journal of Dental Education has been fairly uniform over the past twelve years. Early publications tended to focus on defining competency and exploring its applications in dental and allied dental education1–8,10–14 and on reports of first implementation efforts.15–17 Other themes emerged subsequently, including the role of competencies in curricular design18–21 and in clinic models of care.16,22–24 Assessment has been a consistent element in the competency literature, initially through reports that described outcomes measures that were expressed in terms of competency25–28 and more recently in the search for authentic evaluation methods.29–36 There have been four studies reporting investigations of the learning theory underlying the continuum beginning with novice performance and leading through competency to expertise.1,37–39
|
If competency-based education has been successful, it is also certainly a paradoxical success. All dental schools have curricula that are nominally grounded in competencies. But there appears to be considerable variation in the way this accreditation requirement is implemented. Anecdotally, some schools have replaced "requirements" with "minimal expected procedures" or "thresholds" that function in a similar fashion to numerical graduation requirements. We also believe that there has been a trend in recent years to reduce the number of competencies at schools and to focus them on one-shot clinical test cases that resemble initial licensure examinations. The Commission on Dental Accreditation standards for predoctoral education currently specify a basic set of eight professional (behavioral science) and sixteen clinical competencies. Some schools claim to have as few as ten or twelve.
The intent of this article is to present information on the current understanding among academic and clinical deans and department chairs in U.S. and Canadian dental schools regarding competency-based dental education and to gauge the extent of its application and effects in educational programs.
|
Methods |
|---|
|
TopAbstractMethodsResultsDiscussionConclusionReferences |
|---|
The questionnaire was mailed in mid-September 2005 to the academic and clinical deans and the chairs of the departments of endodontics and restorative dentistry (or their nearest equivalent as determined from the Directory of ADEA Institutional Members) in the sixty-six U.S. and Canadian dental schools. A brief statement of the purpose of the questionnaire, along with instructions, was included. Respondents were asked not to forward the questionnaire to others in their institutions. A follow-up questionnaire was sent to nonrespondents one month later, with a final follow-up in early November.
Descriptive statistics (averages and proportions) were calculated, with chi-square tests, tests for differences in proportions, and Spearman correlation coefficients used as inferential tests to support generalizable conclusions.
|
Results |
|---|
|
TopAbstractMethodsResultsDiscussionConclusionReferences |
|---|
Fewer than half (46 percent) of the respondents were able to identify the conventional definition of competency that was stated as "a set of skills, knowledge, and values that characterize beginning dentists." (See Table 1.) Almost a quarter each identified "competency" as a type of clinical examination format (26 percent) or roughly the same as a clinical discipline (25 percent). Nine percent associated the primary meaning of "competency" as an accreditation requirement.
|
.) The very similar meaning of "all that is taught in didactic and preclinical technique courses" accounted for another 13 percent of choices. Only 5 percent of respondents limited "foundation knowledge" to material covered in didactic courses, correctly recognizing that preclinical technique courses cover foundation knowledge and skills. Seventeen percent reported that "foundation knowledge" is not a notion in current use at their schools.
Table 2 displays respondents’ perceptions of the extent to which six different groups of individuals in dental education were likely to understand and value competency-based education. Accreditation site visitors and dental school administrators were regarded as having the greatest degree of understanding and appreciation of competency-based education. Almost 90 percent of these individuals were thought to have a "high" or "good" understanding and appreciation for competencies. Significantly fewer faculty members and students were regarded as having an understanding and appreciation of competencies. Department chairs represented an intermediate group between accreditation site visitors or administrators compared to faculty members or students. The fact that administrators and accreditation site visitors, department chairs, and faculty and students constitute three distinct groups in terms of their understanding and valuing of competencies was confirmed by chi-square analysis. In this approach, the chi-square test revealed no differences between administrators and accreditation site visitors, but significant differences between these two categories of dental educators and all others did emerge. These two types of educators were thus considered a distinct set. Similar logic was used to classify the other two sets of chairs and of faculty members and students as being distinct. This approach is similar to a post hoc range test in ANOVA, but is adapted in this case for nonparametric data. (The separation of response categories into statically non-overlapping sets on this and other questionnaire items is summarized below in Table 7.) There was a strong correlation between the perception of understanding and the perception of valuing for competency-based dental education across the six groups from whom opinions were solicited (r=.556, p<.001). Those groups regarded as being most knowledgeable were also thought to have the highest appreciation for the value of this educational approach.
|
|
. Both a structured question, inviting respondents to pick all applicable options, and an invitation to offer comments were provided. Twenty-seven percent of respondents indicated that the competency approach had improved the quality of students graduating from dental schools. Almost six in ten respondents pointed to improvements in curriculum management. Competencies are associated with clinical evaluation, and about half of those answering this survey reported satisfaction with the evaluation systems in place, but indicated some ambivalence regarding the use of evaluation information in making decisions about students’ status. Twenty-eight percent stated that conversion to a competency-based educational program is still a work in progress, and often a frustrating one. There were also negative responses: 23 percent said competency is just a new way of talking, not a change in education; 18 percent said it got their schools through accreditation without having actually made changes in their programs; and 10 percent said competency only means extra work. Chi-square tests confirmed that the frequency of reported perceived impact of competencies differed significantly across the categories of positive reported changes, difficulties in implementation, and negative or cosmetic effects.
|
under the headings of curriculum management, clinic changes, evaluation methods, and outcomes. Those who felt there were no changes attributable to a competency approach to dental education naturally offered few comments.
|
shows how the respondents in this survey characterized such testing at their schools. Respondents were required to indicate their perceptions of how competency-based clinical evaluation was implemented at their schools by addressing questions related to timing of completing test cases, determination of passing scores, and consequences of failing test cases. As indicated in Table 5, a number of responses were presented for each of these evaluation factors. Respondents could select all options that applied in their schools. Respondents indicated that students were typically allowed to choose the opportunities that count as test cases, but only during specified windows of time and in accordance with written criteria. Fifty-seven percent of respondents reported that only certain faculty members are allowed to evaluate students’ performance on test cases. Relatively few respondents indicated that faculty members can declare any patient a test case (17 percent) or that test cases could be bunched at specific times, such as near graduation (15 percent).
|
Seventy-seven percent of respondents said that students must continue taking test cases until they pass them, while 27 percent said there is a predetermined maximum number of attempts allowed. On 24 percent of surveys, it was indicated that a committee determines the path forward when students fail to perform in a satisfactory manner on test cases. Automatic remediation was in place for students with multiple test case failures 37 percent of the time. Chi-square analysis confirmed that competency examination administration criteria more often allowed student than faculty control of timing and that open repeated testing opportunities are more common than limited testing or automatic remediation. The least common practice is that no policies exist for timing of test cases or concerning the consequences of failure on such test cases. (See Table 7, below, for a summary.)
Table 6 summarizes the survey results with regard to criteria used by U.S. and Canadian dental schools for determining promotion and graduation of students. Two questions were asked: what is the relative weight of various sources of evidence (must total to 100 percent), and would inadequate performance on any criterion be sufficient by itself to block promotion or graduation (may total to more than 100 percent)? Respondents indicated that multiple sources of information are used and that the weight of these is relatively balanced. Performance on competency test cases was reported as carrying the greatest weight (37 percent), followed by "count data" such as numbers of procedures (22 percent) and numbers of cases completed (13 percent). Faculty judgments were thought to receive slightly less weight: daily work grades (13 percent), faculty grading (8 percent), and committee member judgments (5 percent). Newer forms of evaluation (portfolios and OSCEs) were weighted as less than 1 percent each.
|
, three-quarters of dental schools still retain a "requirement" system. Chi-square analysis reveals that competency test cases play the largest role in determining graduation (both weights and potential to block graduation), followed by numbers of procedures and cases completed, daily work performance, faculty and committee input, and then newer evaluation methods such as OSCEs and portfolios. There was a significant correlation between perceived weight and capacity to block graduation among the evaluation methodologies (r=.345, p<.001). There were also significant correlations between the number of years survey subjects had been in dental education and their responses. Those with more years of experience, for example, were less likely to define foundation knowledge in terms of didactic material only (r=.226) and less likely to believe that numbers of procedures (r=.174) or number of cases completed (r=.159) would block graduation. More experienced respondents were also less likely to report that students had an opportunity to repeat competency test cases until they reached a competent level (r=.221).
The statistically significant differences and correlations for all survey data previously described are summarized in Table 7. In this table, each set of statistically distinct perceptions among respondents is indicated by brackets { }. For example, for the first item in Table 7, definition of foundation knowledge, significantly more respondents defined the concept as "required for competency" than "didactic and preclinical" or "not used," which in turn were significantly more frequent responses than the set containing the two responses "didactic only" and "course objectives."
|
Discussion |
|---|
|
TopAbstractMethodsResultsDiscussionConclusionReferences |
|---|
The accepted definition of competency was recognized by fewer than half of the respondents to this survey. While respondents thought that the concept of competency-based education was understood and valued among administrators and others responsible for curriculum design and innovation, faculty members and students were thought to be less familiar with and less enthusiastic about the approach. Survey responses included detailed recitations of the use of competencies to manage curriculum, stimulate innovation, and even (in the case of one-quarter of respondents) improve the quality of graduates. At the same time, frustration and a "superficial" adoption of competency language for accreditation purposes were also noted among the responses. Almost a third of respondents said that competency-based education has led to no changes.
Despite the requirement that all dental schools must demonstrate complete adoption and use of competency approaches as part of accreditation (and all schools have successfully gone through this process at least once since the early 1990s), the find-ings from this survey suggest that competency-based education may be accurately described as partially implemented. This incomplete adoption is evident on the following dimensions: 1) inconsistent understanding of the construct, 2) differential adoption across schools, 3) better acceptance among administrators than among faculty members and students, 4) different areas of educational programs to which it is applied, and 5) varied perceived impact or benefit. We may accept that there are many cases where competency-based education has led to "more competent students," "better patient care," "better diagnostic and clinical judgment skills," "more curricular integration," and "improved evaluation." But equally valid are the comments that "faculty members have to do more remedial teaching," "some are frustrated," "we just talk different," and "I can’t think of any change that has come as a result of competencies." These differences reflect an irregular application of the policy of competency-based dental education. It is likely that these differences in application and impact represent differences across schools or respondents rather than characteristics of competency-based education itself.
At the same time we must account for the fact that competency has proven to be a very serviceable construct. Those who have studied innovations in education and business generally41–45 agree that six to eight years is a good run in terms of the duration or sustainability of the concept. David and Strang46 studied the Total Quality Management field in business and found that the peak of the literature in this field occurred in 2001. In the three years preceding and the three years following that peak, the number of publications was less than 20 percent of the maximum. Consulting in the field of TQM dropped by similar amounts. By contrast, competency-based dental education continues to enjoy general use in educational programs and generates both new empirical research and reports of program innovations. (See Figure 1.) It probably enjoys more success in dental schools than do PBL, tobacco cessation programs, geriatric dentistry curricula, and evidence-based dentistry (EBD). It also seems to have outlived the results attributable to the Kellogg report,47 Pew report,48 Institute of Medicine report,49 or ADA future of dentistry report.50
The reason for both partial implementation of competency-based dental education and its longevity as a movement may be attributable to the same fact. Competency-based dental education is a protean construct. It is not prescriptive, nor is it a fixed procedure. Instead, it is a point of view or set of basic values about education. Among the elements in this perception of competency are 1) approaching fixed outcomes while allowing flexible means; 2) recognizing that learners progress through various approximations of mastery of a discipline and that each level has its own best way of teaching and evaluation; 3) acceptance of outcomes as the ultimate test of alternative educational processes; and 4) insistence that evaluation be authentic (representative of the situation in which an individual is expected to perform). In sum, the competency perspective places learning at a higher level than teaching in the educational process.
Dental educators who frame their work in terms of improving learning outcomes are likely to report that competency-based dental education has been effective both as a framework for curricular reform and the clinic model and that students’ performance has been improved. For others, this change of perspective is confusing or even threatening. Faculty members who measure education by the quality of what they do (what the faculty member does, e.g., clock hours or their own technical skill) may tend to be frustrated, mystified, or openly resistant to an educational program based on competency. The competency approach points in the direction of integration and student application of diverse learning to specific patient situations rather than procedure- or discipline-segmentation of learning. This aspect of competency spills over into comprehensive care verses discipline clinical models, cross-disciplinary teaching, and elimination of clinical requirements. Chambers’s research on the general competency hypothesis suggests that, regardless of whether faculty members teach discrete segments of dentistry, students learn dentistry generally.37 At the same time, because competency is a perspective rather than a process, its implications for dental education, especially for local application, will take years to work through.
A second paradox concerns competency evaluation. About a third of the literature in competency-based dental education concerns itself with evaluation and assessment. The term "competency" is used in some dental schools to denote a clinical test case rather than a generalized set of skills, knowledge, and professional values that students are supposed to acquire on the way to becoming dentists. It is a name for the test rather than what is being tested.
The results of this survey indicate that clinical test cases have acquired a firm position in dental education. Although they are one among several means of measuring competency, they carry heavy weight for potentially disqualifying students from graduation based on poor performance. Clinical test cases are one of several types of evaluations that are known by the name "authentic evaluation."31 Authentic evaluation requires independent performance, realistic settings, the ability to generalize to comparable varied settings, and professional judgment on the part of the evaluator.
It has been recognized that the number of procedures completed fails to meet the criteria for authentic evaluation; hence, the interest in comprehensive patient care models18 and elimination of requirements.16,22–24 There is also interest in developing other types of authentic evaluation such as ratings,31,32 standardized patients,29 portfolios,34,35 and OSCEs.30,36
The contrast between the U.S. and the Canadian experience on competency evaluation is instructive. In the mid-1990s, representatives of the Canadian education, practice, and licensure communities agreed that competencies would be used as the simultaneous definition of the skills, knowledge, and values required to graduate from dental school and to enter practice. In the United States, the examining community insisted on its own, very narrow, "independent" definition of the skills and knowledge required to begin practice. The perpetual turmoil that divides the profession on licensure issues is a result of failure to accept a competency definition of readiness to begin independent practice.
The retention of number of procedures completed and daily work grades (inauthentic evaluation) side by side with competency approaches to education may reflect a broad discomfort with learning as the measure of student outcomes and a preference for procedure thinking. It may also emanate from discomfort over faculty members having to make and defend professional judgments (rather than relying on counts). A final possibility is that determination of competency is not, after all, a matter of finding the right type of evaluation system. Psychometricians have established that validity is not a matter of using a good test but rather a matter of making a correct decision based on the test.51 This may explain why nearly half of the respondents to the survey reported that "we evaluate fine; the problem is willingness to make the hard decisions about incompetent students."
|
Conclusion |
|---|
|
TopAbstractMethodsResultsDiscussionConclusionReferences |
|---|
|
Footnotes |
|---|
|
REFERENCES |
|---|
|
TopAbstractMethodsResultsDiscussionConclusionReferences |
|---|
This article has been cited by other articles:
|
|
 |
|
  V. Badner, K. P. Ahluwalia, M. K. Murrman, M. Sanogo, T. Darlington, and B. L. Edelstein A Competency-Based Framework for Training in Advanced Dental Education: Experience in a Community-Based Dental Partnership Program J Dent Educ., February 1, 2010; 74(2): 130 - 139. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
D. O. Willis Using Competencies to Improve Dental Practice Management Education J Dent Educ., October 1, 2009; 73(10): 1144 - 1152. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
D. W. Chambers and F. W. Licari Issues in the Interpretation and Reporting of Surveys in Dental Education J Dent Educ., March 1, 2009; 73(3): 287 - 302. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
W. D. Hendricson The ADEA CCI Series of Articles: Perspectives and Reflections in Dental Education (PRIDE) J Dent Educ., February 1, 2009; 73(2): 160 - 165. [Full Text] [PDF]
|
|
|
|
|
|
J. E.N. Albino, S. K. Young, L. M. Neumann, G. A. Kramer, S. C. Andrieu, L. Henson, B. Horn, and W. D. Hendricson Assessing Dental Students' Competence: Best Practice Recommendations in the Performance Assessment Literature and Investigation of Current Practices in Predoctoral Dental Education J Dent Educ., December 1, 2008; 72(12): 1405 - 1435. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
