J Dent Educ. 69(3): 325-337 2005
© 2005 American Dental Education Association
Variation in Periodontal Diagnosis and Treatment Planning Among Clinical Instructors
Sharon K. Lanning, D.D.S.;
Scott D. Pelok, D.D.S.;
Brent C. Williams, M.D., M.P.H.;
Philip S. Richards, D.D.S., M.S.;
David P. Sarment, D.D.S., M.S.;
Tae-Ju Oh, D.D.S., M.S.;
Laurie K. McCauley, D.D.S., Ph.D.
Dr. Lanning is currently Assistant Professor, Department of Periodontics, Virginia Commonwealth University School of Dentistry, and formerly Clinical Assistant Professor, Department of Periodontics, Prevention, and Geriatrics, University of Michigan School of Dentistry; Dr. Pelok is Clinical Assistant Professor, Department of Cariology, Restorative Science, and Endodontics, University of Michigan School of Dentistry; Dr. Williams is Associate Professor, Department of Internal Medicine, University of Michigan School of Medicine; Dr. Richards is Clinical Associate Professor, Department of Periodontics, Prevention, and Geriatrics, University of Michigan School of Dentistry; Dr. Sarment is Clinical Assistant Professor, Department of Periodontics, Prevention, and Geriatrics, University of Michigan School of Dentistry; Dr. Oh is a Clinical Assistant Professor, Department of Periodontics, Prevention, and Geriatrics, University of Michigan School of Dentistry; and Dr. McCauley is Professor and Chair, Department of Periodontics, Prevention, and Geriatrics, University of Michigan School of Dentistry. Direct correspondence and requests for reprints to Dr. Sharon Lanning, 520 North 12th Street, P.O. Box 980566, Richmond, VA 23298-0566; 804-828-7951 phone; 804-828-0657 fax; sklanning{at}vcu.edu.
Key words: faculty development, dental faculty, dental hygiene faculty, periodontology, diagnosis, treatment planning, educational research, student assessment, educational technology, case study
Submitted for publication 09/27/04;
accepted 01/07/05
 |
Abstract
|
|---|
Consistency in clinical decision making may be necessary for reliable assessment of student performance and teaching effectiveness, yet little has been done to examine variation in periodontal diagnosis and treatment planning among dental school faculty. The purpose of this investigation was to examine variation among faculty in diagnosis and management of common periodontal diseases. Twenty-seven clinical instructors (periodontists, general dentists, dental hygienists, and first- and second-year periodontal graduate students) reviewed three web-based cases and answered a brief questionnaire focusing on radiographic interpretation, periodontal diagnosis, and treatment planning. Response rates for the three cases ranged from 62 percent to 70 percent. Clinical instructors’ rating of percent bone loss in the majority of cases varied between three descriptive categories for the same tooth. Greater consistency in periodontal diagnosis was noted within the graduate student group as compared to periodontal and dental hygiene faculty groups. Diagnoses offered for one of the three patients varied between gingivitis and chronic and aggressive periodontitis. Six to nineteen different treatment plans (many with subtle differences) were submitted for each of the three cases. Inter-rater variation was qualitatively more prevalent than intra-rater variation. To our knowledge, this is the first study to document substantial variation among instructors in radiographic interpretation, diagnosis, and treatment planning for common periodontal diseases. Qualitative judgments speculating on the impact of variability among dental school faculty on student performance and patient care can be made but as yet remain unknown. Consistent use of accepted practice guidelines and greater consensus-building opportunities may decrease variation among faculty and enhance dental education.
Inaccuracy and variability in clinical decision making among clinicians have been reported at many academic health care institutions. 1–7 In medicine, Goldman et al.1 reported on the inaccurate diagnosis of acute chest pain by emergency room personnel, and Boom et al.2 found differences among university physicians in the differential diagnosis of jaundice. In restorative dentistry, Mileman et al.5 and Espelid et al.6 found considerable examiner variation in the diagnosis of radiographic caries. Other authors have reported low agreement between dental faculty in treatment decision making of carious lesions7–10 and evaluating clinical serviceability of existing amalgam restorations.11–14 Furthermore, Marbach et al.15 and Shetty et al.16 found low agreement among clinicians in judging bruxism severity and classifying mandibular fractures, respectively.
There is potential for variation among periodontal and preventive faculty in evaluating a patient’s periodontal condition and formulating periodontal diagnoses and treatment options. This is particularly true when the faculty is large and its members have diverse training backgrounds. A number of factors are considered during the process of formulating a periodontal diagnosis, prognosis, and treatment plan, and many of these factors are subjective. This enhances the likelihood of increased variation among clinicians. In addition, varying reports in the literature and rapid dissemination of new knowledge further compound the issue. Persson et al.17 showed differences between university dentists in their responses to a lengthy questionnaire that asked about relevant concepts in periodontology. Not only did responses among faculty members differ, but there were differences between faculty responses and evidence-based literature.
The impact of variation among clinical faculty on student performance remains unknown. Students may benefit from witnessing several ways to approach clinical problems. This type of exposure may enhance their knowledge and technical skills needed to perform different treatment modalities and increase the treatment options they can offer to patients. In contrast, considerable variation among faculty may be detrimental to student learning. Dental and dental hygiene students may model their approach to clinical decision making after their instructors by focusing on faculty-specific strategies for addressing clinical problems rather than learning evidence-based diagnostic and treatment criteria. In one report, students perceived differences between their instructors to affect their clinical progress.18 Furthermore, significant variation among faculty compromises the ability to reliably assess student learning and teaching effectiveness as well as the quality of patient care. Accordingly, the purpose of this investigation was to examine the variation in faculty responses to a series of web-based case exercises regarding interpretation of clinical findings, periodontal diagnosis, and treatment planning.
|
Methods
|
|---|
After obtaining approval from the University of Michigan Institutional Review Board, adult patients and dental school faculty were recruited to participate in a series of web-based case exercises. Three patients, ages twenty-six to forty-nine years, were selected by one author (SKL) primarily because they were representative of undergraduate clinic patients in terms of disease type and severity and similar to the periodontal patients commonly encountered in the practice of general dentistry. In addition, one patient (case 3) was selected because two clinical instructors had previously debated the patient’s periodontal diagnosis. Full- and part-time periodontal (periodontists and general dentists) and dental hygiene faculty and first- and second-year periodontal graduate students from the University of Michigan School of Dentistry were asked to review the web-based cases and answer a questionnaire about the patient’s condition and diagnosis. The faculty and graduate students who participated in this investigation are collectively referred to as "clinical instructors." Each case consisted of a patient’s medical and dental history, description of chief complaint, intra-oral pretreatment photographs, radiographs, and clinical findings. Clinical findings included periodontal charting and written description of extra- and intra-oral evaluation, occlusal and temporomandibular joint examination, and limited radiographic summary. All findings were reported in accordance with the school’s undergraduate clinic protocol.
A twelve-item questionnaire accompanied each case, but this article will describe only five of the questions (Figure 1
). The remaining questions dealt largely with our school’s undergraduate clinic protocol, and the clinical instructors’ answers were not pertinent to the objectives of this study. Questions 1 and 2 asked clinical instructors to identify themselves by their position in the department and years of experience, respectively. Question 3 was an open-ended question that asked clinical instructors to respond with their periodontal diagnosis including a description of extent, severity, and type of disease. Question 4 asked clinical instructors to respond with a periodontal disease control phase treatment plan for the patient including procedure name, code, and quadrant, if indicated. To help answer this question, clinical instructors were referred to a list of twenty-six American Dental Association (ADA)-recognized procedures19 and six school-specific procedures, associated codes, and quadrant notation. For example, it is important to note that the procedure called prophylaxis with gingivitis is intended for patients with abundant local factors requiring lengthy appointments for scaling therapy as opposed to prophylaxis alone, which is a much less involved, in time and extent, therapeutic procedure. Only codes frequently used in periodontal disease control treatment planning were included on this thirty-two item list. Clinical instructors were then asked to quantify the percent bone loss (question 5) associated with index Ramfjord teeth20 (#3, 9, 12, 19, 25, and 28) for all cases except one in which the patient was missing tooth #9. In this case, clinical instructors were asked to quantify the percent bone loss for tooth #10 instead. The following four choices were provided for question 5: no bone loss, less than 15 percent, between 15 percent and 30 percent, and greater than 30 percent. Choices were based on ADA and American Academy of Periodontology (AAP)21–23 guidelines as outlined in the school’s clinic manual for gingivitis and mild, moderate, and severe periodontitis.
Open-ended and extended multiple choice questions minimize cueing and make it possible for examiners to determine if an examinee can generate his or her own response. These types of testing formats are particularly useful in health care where clinicians must be able to answer a clinical question by formulating an original response and not picking it out of a short list of choices. Veloski et al.24 evaluated an open-ended testing format versus multiple choice questions for testing family practice residents on their annual in-service examination. The authors reported higher reliability and validity for an open-ended testing format than multiple choice questions. In this investigation, open-ended and extended multiple choice questions were used for diagnosis and treatment planning questions to gain a better understanding of clinical instructors’ knowledge of diagnostic and treatment criteria. Therefore, neither current AAP diagnostic categories, parameters for defining extent of disease (generalized or localized), nor treatment considerations were reviewed with clinical instructors prior to review of web-based cases and completion of the questionnaire.
Clinical instructors were asked to review case studies and answer the questionnaire independently. The first two cases were electronically posted at the same time, and clinical instructors were given one month to submit their responses online. Responses were presented at a department meeting at which concerns about variability and accuracy of responses and their impact on teaching and student learning were discussed. Two months later, the third case was released, and clinical instructors had five weeks to submit their responses electronically. Response periods of four and five weeks were utilized to allow all clinical instructors ample time to respond.
Web pages were stored on local servers at the School of Dentistry, and access was restricted through a password authentication system. Web page formatting allowed inclusion of multimedia content. One section of the framework included navigation; another, patient information; and a third, radiographic images and digitized charts. The fourth frame enclosed the testing system for the program. Test modules were created using a system that allows anonymous interaction with the modules after authentication is established, thereby ensuring the confidentiality of clinical instructors’ responses.
Two academic periodontists (authors SKL and DPS) thoroughly discussed and defined categories for question 3 based on question construct, a subset of responses randomly selected, and the current AAP classification of periodontal diseases and conditions.22,23 Extent of disease was categorized as generalized or localized. Severity of disease was categorized as mild, moderate, or severe, and common synonyms were included. For example, advanced and severe were included in the same category. Type of disease was categorized as "gingivitis-no descriptor," "plaque-induced gingivitis," "other types of gingivitis," "periodontitis-no descriptor," "chronic or adult periodontitis," "aggressive periodontitis," and "other types of periodontitis." The categories of "gingivitis-no descriptor" and "periodontitis-no descriptor" were included for those responses of gingivitis or periodontitis alone where terms such as plaque-induced or chronic were not given. The categories "other types of gingivitis" and "other types of periodontitis" were added to account for responses not included in the current AAP classification,22,23 such as chronic gingivitis and non-aggressive periodontitis.
In addition, two of the authors (SKL and DPS) discussed and defined categories for question 4 based on question construct, a subset of responses randomly selected, and a listing of recognized ADA19 and school-specific procedures, codes, and quadrant notation as described above. A miscellaneous category was developed for responses not included on the list mentioned above, such as "amalgam restoration." Unconventional quadrant categories were added to account for responses such as "maxillary arch" or "right side."
Two of the authors (SKL and TO) independently assigned responses from questions 3 and 4 into a total of twelve diagnostic and twenty-two treatment planning related categories, respectively. Responses that did not include all diagnostic or treatment planning categories were noted as such. Agreement between categorized data was evaluated by inter-rater statistic (Cohen’s Kappa [
]). Responses from question 5 were analyzed using generalized Fischer’s exact test by position in the department and years of clinical experience. A P value of <0.05 was considered significant.
|
Results
|
|---|
Response rates for the three cases are presented in Table 1. Graduate students had the lowest response rate for all three cases while periodontal faculty had the highest response rate for cases 1 and 2 at 78 percent and 83 percent, respectively. Eight dental hygiene faculty members (73 percent) responded to all three cases. Overall, the response rate ranged between 62 and 70 percent for the three cases. Graduate students were the only group with less than five years of clinical experience. Over 65 percent of periodontal and 88 percent of dental hygiene faculty members had greater than ten years of clinical experience.
Responses submitted for questions 3 and 4 were assigned independently by two raters (SKL and TO) into a total of twelve diagnostic and twenty-two treatment planning related categories with high agreement ( = 0.93 and = 0.89), respectively. Responses from question 5 revealed no significant differences when analyzed by the generalized Fischer’s exact test by position in the department and years of clinical experience.
|
Case 1
|
|---|
Of the twenty-seven clinical instructors who responded, 52 percent submitted a diagnosis including extent, degree, and type of disease. Fifty-five percent of clinical instructors reported the disease to be generalized. Degree ranged between mild gingivitis and moderate periodontitis. Twenty-five percent of clinical instructors submitted mild periodontitis to describe periodontitis alone (no descriptor) or chronic periodontitis. The types of disease included gingivitis-no descriptor, plaque-induced gingivitis, gingivitis-other (chronic gingivitis), periodontitis-no descriptor, chronic periodontitis, aggressive periodontitis, and periodontitis-other (non-aggressive periodontitis) (Table 2). The most common diagnosis was gingivitis-no descriptor offered by 48 percent of clinical instructors. Overall, 67 percent of clinical instructors diagnosed the patient with some type of gingivitis, while the remaining 33 percent offered periodontitis.
All graduate students diagnosed the patient with gingivitis and recommended prophylaxis with gingivitis (Table 3). Ten (71 percent) periodontal faculty members submitted the diagnosis of gingivitis and offered four different treatment plans including full mouth debridement, prophylaxis, prophylaxis with gingivitis, and four full quadrants of scaling and root planing. Four (29 percent) periodontal faulty members diagnosed the patient with periodontitis. There were three different treatment plans submitted including one full quadrant of scaling and root planing plus periodontal maintenance, four full quadrants of scaling and root planing, and four less than full quadrants of scaling and root planing. Three dental hygiene faculty members (38 percent) offered a diagnosis of gingivitis and recommended prophylaxis with gingivitis. Five (63 percent) dental hygiene faculty members submitted a diagnosis of periodontitis; their treatment plans included full mouth debridement, prophylaxis, prophylaxis with gingivitis, and four full quadrants of scaling and root planing. Overall, the most common treatment plan was prophylaxis with gingivitis submitted by 59 percent of clinical instructors. The second most common treatment plan was four full quadrants of scaling and root planing offered by 15 percent of clinical instructors.
Sixty-three percent of clinical instructors selected no bone loss for tooth number 3, while the remaining 37 percent chose <15 percent bone loss (Figure 2). All of the graduate students and the majority of periodontal and dental hygiene faculty submitted no bone loss for tooth numbers 9, 12, 19, and 28. The distribution of responses ranged between none and 15–30 percent bone loss. The periodontal faculty group had the greatest distribution of responses for tooth numbers 12, 19, 25, and 28. For tooth number 25, 48 percent, 44 percent, and 7 percent of clinical instructors selected none, <15 percent, and 15–30 percent bone loss, respectively. For tooth number 28, 74 percent, 24 percent, and 4 percent of clinical instructors selected none, <15 percent, and 15–30 percent bone loss, respectively. The periodontal faculty had the greatest distribution of responses. Of the eighteen clinical instructors that diagnosed the patient with gingivitis, 40 percent of graduate students, 30 percent of periodontal, and 100 percent of dental hygiene faculty members submitted one or more index Ramfjord teeth with bone loss (Table 4). One periodontal faculty member determined the bone loss to be 15–30 percent bone loss for two teeth.
|
Case 2
|
|---|
Ninety-six percent of clinical instructors included extent, degree, and type of disease in their diagnosis. Eighty-two percent of clinical instructors reported the disease to be generalized, with the degree of severity ranging from moderate gingivitis to severe periodontitis. Fifty-five percent of clinical instructors submitted moderate periodontitis. Degree ranged from moderate gingivitis to severe periodontitis. Fifty-five percent of clinical instructors submitted moderate periodontitis. The types of disease included gingivitis-other (chronic gingivitis), periodontitis-no descriptor, chronic periodontitis, and periodontitis-other (non-aggressive and reversible periodontitis) (Table 5). The most common diagnosis was chronic periodontitis offered by 67 percent of clinical instructors. Overall, 96 percent of clinical instructors diagnosed the patient with some type of periodontitis.
In total, nineteen different treatment plans were submitted for case 2. The most common treatment submitted was four full quadrants of scaling and root planing offered by 11 percent of graduate students and 18 percent of periodontal faculty. The four most common procedures were four full quadrants of scaling and root planing, fabrication of a bite splint, limited occlusal adjustment, and four less than full quadrants of scaling and root planing recommended by 75 percent, 34 percent, 21 percent, and 18 percent of clinical instructors, respectively (Table 6). Other recommended treatment included full mouth debridement, prophylaxis with gingivitis, locally delivered antimicrobials, and complete occlusal adjustment.
Bone loss responses for tooth numbers 10, 12, and 28 were distributed between no bone loss and greater than 30 percent (Figure 3). For tooth number 10, the majority of graduate students selected bone loss between 15 and 30 percent, the majority of periodontal faculty members chose <15 percent, and the majority of dental hygiene faculty members selected no bone loss. Overall, 43 percent of clinical instructors submitted between 15 and 30 percent bone loss for tooth number 12; and 52 percent, 42 percent, and 44 percent of clinical instructors submitted bone loss <15 percent for tooth numbers 19, 25, and 28, respectively.
|
Case 3
|
|---|
Ninety-two percent of clinical instructors included extent, degree, and type of disease in their diagnosis. Seventy-one percent of clinical instructors reported the disease to be generalized, with the degree of severity ranging from mild gingivitis to severe periodontitis. Fifty-eight percent of clinical instructors submitted mild periodontitis. The types of disease included gingivitis-no descriptor, periodontitis-no descriptor, chronic periodontitis, and aggressive periodontitis (Table 7). The most common diagnosis was chronic periodontitis offered by 50 percent of clinical instructors. Overall, 12 percent of clinical instructors diagnosed the patient with some type of gingivitis, while the remaining 88 percent listed periodontitis as the diagnosis.
Half of the clinical instructors recommended four full quadrants of scaling and root planing and extraction of the third molars, which was the most common treatment plan. In total, seven different treatment plans were submitted. The most common procedure was four full quadrants of scaling and root planing recommended by 100 percent of graduate students, 83 percent of periodontal faculty, and 75 percent of dental hygiene faculty members (Table 8). Fifty-eight percent of treatment plans called for extraction of the third molars. One dental hygiene faculty member submitted a diagnosis of gingivitis and three separate prophylaxis procedures and one full quadrant of scaling and root planing.
Bone loss responses for tooth numbers 3, 19, 25, and 28 were between no bone loss and 15–30 percent and for tooth numbers 9 and 12 between no bone loss and <15 percent (Figure 4). The periodontal faculty members had the greatest distribution of responses. Overall, the most common response for tooth numbers 3, 12, 19, 25, and 28 was <15 percent bone loss by 65 percent, 86 percent, 78 percent, 60 percent, and 70 percent of clinical instructors, respectively. Sixty-nine percent of clinical instructors indicated that the patient in case 3 had no bone loss for tooth number 9.
|
Discussion
|
|---|
This investigation demonstrated variations in clinical instructors’ responses to a series of web-based cases representing undergraduate clinic patients in terms of disease type and severity. Clinical instructors’ rating of percent bone loss in the majority of cases varied among three descriptive categories for the same tooth. Diagnoses submitted for the patient represented in case 1 included gingivitis and chronic and aggressive periodontitis. Four different treatment plans were offered for the diagnosis of gingivitis alone. Nineteen different treatment recommendations were submitted for the patient in case 2. In addition, our results also showed intra-rater discrepancies. Forty-four percent of clinical instructors that diagnosed patient case 1 with gingivitis submitted one or more index Ramfjord teeth with bone loss. Two clinical instructors recommended scaling and root planing for cases they diagnosed with gingivitis (cases 1 and 3).
Caution must be taken when counting the number of different treatment recommendations offered for the same patient as true differences in treatment or treatment outcomes. A "prophylaxis" or "prophylaxis with gingivitis" and even "full mouth debridement" may be, in actual practice, quite similar. In addition, the patients’ response to these treatments may not be significantly different. However, the cost, reimbursement, treatment time, and student understanding of the technical differences and rationale for these procedures may be the consequences of these different treatment recommendations. Lastly, the variety of treatment plans submitted in this investigation may be attributed to instructors using terminology loosely whereas their intended treatment for a given patient was in fact the same.
Some degree of variation among clinicians is expected. There are a number of elements that go into generating a periodontal diagnosis and treatment approach. Although evidence-based literature and accepted practice guidelines are used to support clinical decision making, there are a number of subjective factors involved in the practice of periodontics as is the case with other areas of medicine and dentistry.13,14,25 In the area of treatment, greater variation is expected and acceptable as long as it is supported by evidence from clinical trials and consistent with established practice guidelines. Conversely, differences among clinicians in determining percent bone loss and diagnoses are expected to a lesser degree. Determining percent bone loss is based on the relationship between anatomical factors that can actually be measured.26 Therefore, determining percent bone loss is quite objective. This is not to say variations in judging the degree of disease are unexpected—say, from mild to moderate periodontitis. However, diagnoses for the same patient that include gingivitis and aggressive periodontitis are less expected and acceptable since these diagnoses have very different contributing factors, clinical presentations, prognoses, and treatment considerations.22
As the number of factors that go into formulating a periodontal diagnosis and treatment plan increases, so does the likelihood of inaccuracy and variability in clinical decision making among clinicians. This is particularly true when there are a number of subjective elements. For example, the diagnosis of gingivitis is largely based on gingival appearance.23 Lack of consistency among clinicians may also be a result of recognition error and unfamiliarity with the use of accepted criteria.27 Bader and Shugars10 wrote that "disagreements [among clinicians] may be due to differing degrees of diagnostic thoroughness or strongly held personal opinions about appropriate treatment." In addition, the rapid dissemination of new knowledge also makes it more difficult for clinicians to be up-to-date. Inconsistent reports in the literature can also contribute to variability in treatment decision making among clinicians. Furthermore, a large diverse teaching body may be more prone to variability among its members, especially when there is little opportunity to meet and discuss cases and stress the use of accepted diagnostic and treatment criteria. Graduate student responses for some aspects of the cases were more consistent than responses from periodontal and dental hygiene faculty. Since graduate students meet frequently in seminars and case conferences, they are exposed to graduate faculty and each other’s interpretation of clinical findings and therapeutic philosophies. Hence, these sessions serve as built-in consensus-building opportunities.
This investigation was not designed to give the "right answer" or even an acceptable range of "right answers" against which clinical instructors’ responses were compared. However, based on the vast range of responses, there are clearly outliers and inaccuracies. For example, bone loss determinations for the same radiograph (case 2, tooth #10, 12, and 28) consisted of none, <15 percent, 15–30 percent, and >30 percent. Diagnoses offered for the same patient (case 1) included gingivitis and chronic, aggressive, and non-aggressive periodontitis. Disease control phase therapy for the same patient (case 3) diagnosed with gingivitis consisted of prophylaxis, full mouth debridement, and scaling and root planing.
Accuracy consists of reliability (getting the same answer when you measure the same thing more than once) and validity (getting the right answer, reaching the "truth").28 Validity is often measured by comparing a student’s answer to a gold standard. The gold standard is commonly the opinion of the faculty (i.e., the said expert). Therefore, a good measure of accuracy is reliability as indicated by variability among expert opinion. Thus, if faculty members are not providing a constant target toward which students are to aim, the ability to differentiate between accuracy and inaccuracy may be lost. As a result, the ability to reliably assess teaching and student learning may be compromised.29
Variability among faculty members has been well documented in both medicine and dentistry.1–7 Although variation among faculty members may affect the ability to reliably assess student learning and teaching effectiveness, as explained above, there is little information on the impact of this on student performance and patient care. It can be argued that students who are exposed to different strategies for addressing clinical problems may be better able to decipher unusual clinical presentations of disease and provide greater treatment options to their patients than students exposed to strict interpretation of diagnostic guidelines and limited treatment modalities. In contrast, variability among faculty members may confuse the novice learner and make it difficult for them to interpret clinical findings, integrate concepts, and manage patients.
Approximately a third of clinical instructors diagnosed the web-based cases with gingivitis or periodontitis alone without descriptors. It could be assumed that clinical instructors may have intended these diagnoses to be plaque-induced gingivitis or chronic periodontitis because they are the most common periodontal diseases, respectively. However, question 3 specifically asked for the type of disease; therefore, when evaluating the instructors’ responses, these diagnostic categories were not combined based on "assumption" because raters cannot guess the clinical instructors’ intent.
Current AAP diagnostic criteria are based largely on loss of attachment.23 The web-based cases did not include these data since the cases were modeled after current undergraduate clinic protocol. Without regularly collecting attachment level data, faculty and students alike will have difficulty applying AAP guidelines for periodontitis.
There appears to be less intra- and inter-rater variability from case 1 and 2 to case 3 as illustrated by a smaller distribution of responses for determining percent bone loss and a greater percentage of clinical instructors’ treatment recommendations consistent with their diagnoses. This may be a result of the presentation and discussion of responses for cases 1 and 2 at a faculty meeting or due to inherent case characteristics.
Dental hygienists and graduate students were included in this investigation because they are directly involved with clinical instruction of dental and dental hygiene students for at least one session per week in the teaching clinics and therefore contribute significantly to students’ clinical experience. Dental hygienists are not designated as periodontal disease diagnosticians. However, they are trained to assess a patient’s oral condition, to provide patient education, and to make limited treatment recommendations based on a patient’s medical and dental histories, diagnosis, risk factors, response to past treatment, and clinical findings. Therefore, hygienists must interpret clinical findings and have a fundamental understanding of periodontal diagnostic criteria and treatment modalities. Statistical analysis did not indicate significant differences among graduate students, periodontal faculty, and dental hygiene faculty based upon position in department or years of clinical experience. This suggests that the type of training and clinical experience may not have had a significant influence on clinical instructors’ responses. However, due to the small sample size, we are cautious to draw this conclusion because of the possibility of a type II error. In this investigation, distinctions were not made between part-time or full-time general dentists or periodontal faculty members and graduate students in order to minimize the identification of any one clinical instructor.
This investigation could be used as a model for examining the variability among faculty in a number of disciplines. The web-based design made it easy for clinical instructors to navigate through the cases and submit their responses anonymously. The response rate ranged from 67 to 83 percent for periodontal and dental hygiene faculty, reflecting not only these positive aspects of the web-based design but also their interest and dedication to the improvement of dental education. There are also negative aspects to web-based design. Clinical instructors could have discussed the cases prior to completing the online questionnaires. Variation in radiographic interpretation could have been due to enlarged nonstandardized image projection via computer monitors or use of digitized images since most of the clinical instructors are not accustomed to viewing digital radiographs. This investigation has other limitations. There was a lower response rate for the graduate student group. The use of web-based design rather than in vivo examination of patients may have contributed to the variability among clinical instructors. It is possible that diagnoses and treatment recommendations could have been more consistent if actual patients were examined. Furthermore, this investigation was not designed to determine accuracy of instructors’ responses. Steps are being made to evaluate accuracy and intra- and inter-rater variability through the development of instructor in-service training sessions.
|
Conclusion
|
|---|
This investigation demonstrated significant variability among preventive and periodontal faculty members and periodontal graduate students regarding interpretation of clinical findings, periodontal diagnosis, and treatment planning. Some degree of variation among clinicians is expected for reasons discussed earlier. Within our dataset there were "outliers," and clearly not all clinical instructors’ responses were accurate. In education, students are assessed on their ability to generate the "correct answer." Their answer is commonly compared to the gold standard, which is the opinion of the faculty. If faculty members are providing an ever-moving target, students’ ability to differentiate between accuracy and inaccuracy could be lost. As a result, student assessment and evaluation of educational goals may be impaired. The impact of variability among faculty on student development and performance remains unclear. Our results suggest that there is room for improvement. An obvious place to begin is consistent utilization of accepted practice and evidence-based guidelines and providing greater opportunities for consensus-building efforts among instructors so that students and educational programs can be appropriately assessed.
|
Acknowledgments
|
|---|
The authors would like to thank Mrs. Barbara Wolfgang and Mrs. Beverly Sutton for administrative assistance, Professor Karen Ridley for assistance with web-base case preparation, Dr. Rodrigo Neiva for categorizing responses, Dr. James Fitzgerald and Ms. Linping Duan for statistical support, and Dr. Larry Gruppen and Mrs. Allison Carey for assistance with manuscript preparation. In addition, the authors would like to acknowledge the periodontal and preventive clinical faculty and graduate students at the University of Michigan School of Dentistry Department of Periodontics, Prevention, and Geriatrics for their participation and dedication to this project.
|
REFERENCES
|
|---|
- Goldman L, Weinberg M, Weisberg M, Olshen R, Cook EF, Sargent RK, et al. A computer-derived protocol to aid in the diagnosis of emergency room patients with acute chest pain. N Engl J Med 1982;307(10):588–96.[Abstract]
- Boom R, Gonzalez C, Fridman L, Ayala JF, Realpe JL, Morales P, Quintero R. Looking for "indicants" in the differential diagnosis of jaundice. Med Decis Making 1986;6(1):36–41.
- Todd BS, Stamper R. Limits to diagnostic accuracy. Med Inform 1993;18(3):255–70.
- Mileman PA, Pudell-Lewis D, van der Weele L. Effect of variation in caries diagnosis and degree of caries on treatment decisions by dental teachers using bitewing radiographs. Community Dent Oral Epidemiol 1983;11(6): 356–62.[Medline]
- Mileman P, Purdell-Lewis DJ, Dummer P, van der Weele L. Diagnosis and treatment decisions when using bite-wing radiographs—a comparison between two dental schools. J Dent 1985;13(2):140–51.[Medline]
- Espelid I, Tveit AB, Fjelltveit A. Variations among dentists in radiographic detection of occlusal caries. Caries Res 1994;28(3):169–75.[Medline]
- Mileman PA, van der Weele LT. Accuracy in radiographic diagnosis: Dutch practitioners and dental caries. J Dent 1996;18(3):130–6.
- Mileman PA, Pudell-Lewis, van der Weele LT. Variation in radiographic caries diagnosis and treatment decision among university teachers. Community Dent Oral Epidemiol 1982;10(6):329–44.[Medline]
- Rytomaa I, Jarvinen V, Jarvinen J. Variation in caries recording and restorative treatment plan among university teachers. Community Dent Oral Epidemiol 1979;7(6): 335–9.[Medline]
- Bader JD, Shugars DA. Agreement among dentists’ recommendation for restorative treatment. J Dent Res 1993;72(5):891–6.[Abstract/Free Full Text]
- Robertello FJ, Pink FE. The effect of a training program on the reliability of examiners evaluating amalgam restorations. Oper Dent 1997;22(2):57–65.[Medline]
- Mahler DB, Engle JH, Bryant RW. Standardizing evaluations of the clinical marginal fracture of amalgam. J Dent Res 1986;65(8):1108–11.[Abstract/Free Full Text]
- Swallow JN, van Groenestijn MA, Maas-de Waal CJ, Mileman PA. Examiner variability in the assessment of dental restorations. Community Dent Oral Epidemiol 1978;6(5):256–63.[Medline]
- Nuckles DB, Sneed WD, Bayme JB, Collins DE, Hook CR, Welsh EL. Faculty differences in replacement decisions for amalgam restorations. Quintessence Int 1991;22(7):533–40.[Medline]
- Marbach JJ, Raphael KG, Janal MN, Hirschkorn-Roth R. Reliability of clinician judgments of bruxism. J Oral Rehabil 2003;30(2):113–8.[Medline]
- Shetty V, Atchison K, Der-Martirosian C, Wang J, Belin TR. Determinants of surgical decisions about mandible fractures. J Oral Maxillofac Surg 2003;61(7):808–13.[Medline]
- Persson GR, Schlegel-Bregenzer B, Lang NP, Attstrom R. Education in periodontology: a need for a new teaching model. Eur J Dent Educ 1999;3(2):74–81.[Medline]
- Stach DJ, Cross-Poline GN. Clinical faculty calibration using case-based exercises with videotapes. J Dent Educ 2002;66(2):289.
- Current dental terminology. Chicago: American Dental Association, 2003.
- Ramfjord SP. Indices for prevalence and incidence of periodontal disease. J Periodontol 1959;30:51–9.
- Cameron CA, Perry DA. Clinical assessment. In: Perry DA, Beemsterboer P, Taggart EJ, eds. Periodontics for the dental hygienist, 2nd ed. Philadelphia: W.B. Saunders, 2001:108–9.
- Armitage GC. Development of a classification system for periodontal diseases and conditions. Ann Periodontol 1999;4(1):1–6.[Medline]
- Parameter on periodontal maintenance. J Periodontol 2000;71(5 Suppl):847–80.[Medline]
- Veloski JJ, Rabinowitz HK, Robeson MR, Young PR. Patients don’t present with five choices: an alternative to multiple-choice tests in assessing physicians’ competence. Acad Med 1999;74(5):539–46.[Medline]
- Brooker C, Kamien M, Ward AM. Differences in teaching about the acute sore throat within one medical faculty. Med Educ 2000;34(4):269–74.[Medline]
- Schei O, Waerhaug J, Lovdal A, Arno A. Alveolar bone loss related to oral hygiene and age. J Periodontol 1959;30:7–16.
- Goldstein IL, Mosbley WH, Chellemi SJ. The observer process in the visual interpretation of radiographs. J Dent Educ 1971;35:485–91.[Medline]
- Linn RL, Gronlund NE. Measurements and assessment in teaching, 8th ed. Upper Saddle River, NJ: Prentice-Hall, 2000.
- Rowntree D. Assessing students: how shall we know them? 2nd ed. Sterling, VA: Stylus Publications, 1987.
This article has been cited by other articles:
|
|
|
|
 
K. V. Garland and K. J. Newell
Dental Hygiene Faculty Calibration in the Evaluation of Calculus Detection
J Dent Educ.,
March 1, 2009;
73(3):
383 - 389.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
J. E.N. Albino, S. K. Young, L. M. Neumann, G. A. Kramer, S. C. Andrieu, L. Henson, B. Horn, and W. D. Hendricson
Assessing Dental Students' Competence: Best Practice Recommendations in the Performance Assessment Literature and Investigation of Current Practices in Predoctoral Dental Education
J Dent Educ.,
December 1, 2008;
72(12):
1405 - 1435.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
M. E. Jacks, C. Blue, and D. Murphy
Short- and Long-Term Effects of Training on Dental Hygiene Faculty Members' Capacity to Write SOAP Notes
J Dent Educ.,
June 1, 2008;
72(6):
719 - 724.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
A. D. Padbury Jr., T. F. Tozum, M. Taba Jr., E. L. Ealba, B. T. West, R. E. Burney, P. G. Gauger, W. V. Giannobile, and L. K. McCauley
The Impact of Primary Hyperparathyroidism on the Oral Cavity
J. Clin. Endocrinol. Metab.,
September 1, 2006;
91(9):
3439 - 3445.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
S. K. Lanning, A. M. Best, H. J. Temple, P. S. Richards, A. Carey, and L. K. McCauley
Accuracy and Consistency of Radiographic Interpretation Among Clinical Instructors in Conjunction with a Training Program
J Dent Educ.,
May 1, 2006;
70(5):
545 - 557.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
S. K. Lanning, A. M. Best, H. J. Temple, P. S. Richards, A. Carey, and L. K. McCauley
Accuracy and consistency of radiographic interpretation among clinical instructors using two viewing systems.
J Dent Educ.,
February 1, 2006;
70(2):
149 - 159.
[Abstract]
[Full Text]
[PDF]
|
|
|
Top
Abstract
Methods
