JDE
HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS
QUICK SEARCH:   [advanced]


     

J Dent Educ. 70(2): 149-159 2006
© 2006 American Dental Education Association
This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Lanning, S. K.
Right arrow Articles by McCauley, L. K.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Lanning, S. K.
Right arrow Articles by McCauley, L. K.

Critical Issues in Dental Education

Accuracy and Consistency of Radiographic Interpretation Among Clinical Instructors Using Two Viewing Systems

Sharon K. Lanning, D.D.S.; Al M. Best, Ph.D.; Henry J. Temple, D.D.S.; Philip S. Richards, D.D.S., M.S.; Allison Carey, B.S., M.P.H.; Laurie K. McCauley, D.D.S., Ph.D.

Key words: radiographic interpretation, periodontology, faculty development, dental faculty, dental hygiene faculty, educational research, student assessment

Submitted for publication 06/20/05; accepted 10/24/05


   Abstract
Top
 Abstract
Methods
 Results
 Discussion
 Conclusion
 References
 
Accurate and consistent radiographic interpretation among clinical instructors is needed for assessment of teaching, student performance, and patient care. The purpose of this investigation was to determine if the method of radiographic viewing affects accuracy and consistency of instructors’ determinations of bone loss. Forty-one clinicians who provide instruction in a dental school clinical teaching program (including periodontists, general dentists, periodontal graduate students, and dental hygienists) quantified bone loss for up to twenty-five teeth into four descriptive categories using a view box for plain film viewing or a projection system for digitized image viewing. Ratings were compared to the correct category as determined by direct measurement using the Schei ruler. Agreement with the correct choice for the view box and projection system was 70.2 percent and 64.5 percent, respectively. The mean difference was better for a projection system due to small rater error by graduate students. Projection system ratings were slightly less consistent than view box ratings. Dental hygiene faculty ratings were the most consistent but least accurate. Although the projection system resulted in slightly reduced accuracy and consistency among instructors, training sessions utilizing a single method for projecting digitized radiographic images have their advantages and may positively influence dental education and patient care by enhancing accuracy and consistency of radiographic interpretation among instructors.

Radiographs are an essential adjunct to the clinical examination for formulating periodontal diagnoses and prognoses and evaluating treatment outcomes.1 Several factors such as widening of periodontal ligament space, crestal bone irregularity, loss of continuity, and height are useful in determining the extent and severity of disease and monitoring the patient’s longitudinal outcomes of therapy.2 The relationship between the crestal bone and cementoenamel junction (CEJ) is commonly used to determine the linear degree of interproximal alveolar bone loss.3 Several authors have investigated the relationship between CEJ and alveolar crestal bone in adolescents and found a distance greater than 2 mm indicates bone loss.4–7

Accurate and consistent identification of crestal bone and its relationship to the CEJ is important for initial diagnosis and long-term evaluation of the periodontium. Previous work has revealed inaccuracy and variability among dental school faculty in quantifying interproximal bone loss.8 This is particularly problematic in an academic setting where patients may be monitored by several dental school faculty during the course of their periodontal therapy and faculty members are responsible for teaching and assessing students’ radiographic interpretation.

The manner by which radiographs are viewed may impact the determination of bone loss. Inaccurate and inconsistent ratings of bone loss among clinical instructors have been reported previously using multiple computer monitors for displaying digitized radiographic images.8 This may have been a result of nonstandardized image projection via computer monitors or use of digitized radiographic images since viewing radiographs in this way is not what most clinicians are accustomed to. Most clinical instructors use a view box to view plain films in the dental setting. It may be that any viewing system that varies from what clinicians are accustomed to could affect the accuracy and consistency of their ratings. In this investigation existing plain film radiographs meeting specific criteria and digitized images of the plain films were viewed using a view box and single LCD projector, respectively. The use of a single method for projecting digitized images offered the advantage of standardized image projection during a group teaching session. Therefore, the purpose of this investigation was to determine if the method of radiographic viewing affected the accuracy and consistency of clinical instructors’ rating of percent bone loss.


   Methods
Top
 Abstract
 Methods
Results
 Discussion
 Conclusion
 References
 
After the University of Michigan Institutional Review Board approved the study protocol, eighty-six plain film periapical radiographs were obtained from either faculty members’ teaching materials or patients’ records at the University of Michigan, School of Dentistry. Two of the authors (SKL and HJT) selected radiographs consistent with Prichard’s criteria of accurate radiographs.9 Specifically, radiographs displayed distinct enamel caps and pulpal chambers, molar cusps with little or no occlusal surface showing, and interproximal contacts free of overlap. Additionally, the CEJ and apex or apices of the study teeth were clearly visible. Radiographs were duplicated using Kodak duplicating film and processor Rp X-OMAT Model M7B with EK Developer Solution and SUREX RP Fixer. The same two authors judged sixty-two radiographs to be of acceptable quality following duplication. Radiographs were individually mounted using clear x-ray mounts and viewed via standard view box in an artificially lit room by two experienced periodontists. Fifty-eight radiographs were judged to possess the characteristics described above and be of acceptable quality. A total of twenty-five radiographs were randomly selected to be used for this study. Radiographs were prepared for projection by scanning them using a flatbed Microtek ScanMaker 8700 scanner and software ScanWizard Pro 7.0, which used a scanning resolution of 300 pixels per inch. Digitized images were imported into Microsoft PowerPoint and projected via LCD projector using a resolution of 1024 x 768 in a dimly lit room. Magnified projected images were judged to be of acceptable quality by the same two authors after minor grey scale adjustments.

The "actual" amount of bone loss was determined by three authors (SKL, HJT, and PSR), independently, on plain film radiographs via standard view box in an artificially lit room using a Schei ruler to the nearest 5 percent.10 The Schei ruler used was a plastic transparent ruler with a 2 mm thick marking at its margin and a series of equidistant lines radiating from a center point, each representing 5 percent bone loss. The ruler was placed on the tooth in question with the 2 mm thick marking at the tooth’s radiographic CEJ; then the ruler was moved until the last radius covered the tooth’s radiographic apex or in the case of teeth with multiple roots the tooth’s most apically positioned radiographic apex. That is, bone loss for multirooted teeth was determined using the tooth’s longest root. The "actual" amount of bone loss was determined by identifying the position of the alveolar bone crest relative to the ruler’s markings. The level of the bone crest was deemed to be at the point along the root’s surface where an intact lamina dura was observed. Teeth with mesial and distal percent bone loss discrepancies were judged by the greater percentage of the two. One discrepancy in rating bone loss occurred among the authors and was discussed until consensus was reached. A computer-generated grid representing the bone loss categories as described below was created and superimposed on the teeth to verify the correct choice category on digitized images independently by two authors (SKL and HJT).

Radiographs included anterior and posterior teeth, maxillary and mandibular teeth, single- and multirooted teeth, teeth with no bone loss, and others with various amounts of bone loss (Table 1Go). Teeth were grouped into four descriptive categories based on severity of bone loss. These categories included bone loss of none, <15 percent, 15–30 percent, and >30 percent based on American Dental Association (ADA) and American Academy of Periodontology (AAP)10–13 guidelines as outlined in the school’s clinic manual for gingivitis, mild, moderate, and severe periodontitis, respectively. For the purpose of statistical analysis, numbers were assigned to each category as follows: none, <15 percent, 15–30 percent, and >30 percent. The first twenty teeth were viewed by both the view box and projection system. All five teeth in the none category had 0 percent bone loss. The three teeth in category <15 percent were determined to have an actual bone loss at 10 percent. Of the six teeth in category 15–30 percent, four had actual bone loss at 15 percent, one had actual bone loss at 20 percent, and one had actual bone loss at 30 percent. Three of the six teeth in category >30 percent bone loss had actual bone loss at 35 percent, two had actual bone loss at 40 percent, and one at 50 percent. The projection system used twenty-five teeth. Added teeth included one tooth at 0 percent bone loss, two at 5 percent (category <15 percent), one at 10 percent (category <15 percent), and one at 25 percent (category 15–30 percent). Teeth viewed during both occasions were paired for purposes of statistical analysis.


View this table:
[in this window]
[in a new window]
  		Table 1. Percent of teeth in sample
 
Full- and part-time periodontal (periodontists and general dentists) and dental hygiene faculty and periodontal graduate students from the University of Michigan School of Dentistry were asked to view the twenty radiographs mounted in a clear x-ray mount using a standard view box in an artificially lit room and to answer a brief questionnaire. These faculty members and graduate students will be collectively referred to as "clinical instructors."

The first item on the questionnaire asked clinical instructors to identify themselves as dental hygiene faculty, graduate student, or periodontal faculty. The second question asked them to quantify their years of clinical experience as <5, 5–10, or >10 years. The remaining questions asked clinical instructors to quantify percent bone loss for indicated teeth as none, <15 percent, 15–30 percent, and >30 percent bone loss for the reasons described above. Written instructions were provided to ensure consistent viewing practices among clinician instructors. Clinical instructors were instructed to determine bone loss from 2 mm apical from the CEJ to the root apex. Teeth with mesial and distal percent bone loss discrepancies were to be rated by the greater percentage of the two. Over a three-week period, clinical instructors independently viewed radiographs and completed the questionnaire.

Four weeks later the clinical instructors viewed digitized radiographic images given in random order as a group and individually answered the questionnaire again. The two occasions of radiographic viewing were scheduled four weeks apart to optimize clinical instructors’ participation given their other teaching, clinical, and research responsibilities. Written and verbal instructions again ensured consistent viewing practices among clinician instructors, and percent bone loss was determined in the same manner as before. For each question, the instructors were given at least thirty seconds to record their response on the questionnaire and transmit their response via wireless remote. An audience response system using the remote was used to capture their responses, which were submitted anonymously either on paper or via wireless remote. Discrepancies between written and transmitted responses were omitted from the research database. Correct choices were presented to the instructors only after observing the radiographs by both viewing systems.

The Kappa coefficient described both agreements between the two viewing systems and accuracy (agreement with the correct choice). While sensitivity and specificity are typically used as indices of accuracy, they are not defined in situations with more than two categories. Accuracy was also measured by differences from the correct choice in two ways. One dependent variable was the difference between the clinical instructors’ rating and the correct choice; this variable is indicated as "difference" in all tables. This difference is thus the signed rater error and reflects net deviation from the correct choice in one direction. A positive difference indicates an overestimation of bone loss, and a negative difference indicates underestimation of bone loss. The second dependent variable used in the final analysis was the absolute value of this difference. A zero indicates a correct choice, and a positive value reflects overall deviation from the correct choice in either direction. This variable is indicated as "absolute" in all tables. Both the arithmetic difference and absolute difference are necessary because there may be zero average difference while the absolute difference is non-zero, and if there is non-zero absolute difference, it is necessary to describe the direction of the difference. Disagreement was analyzed using repeated-measured, mixed-models analysis with the following effects in the ANOVA model: three clinical instructor groups, four correct choice categories, twenty-five radiographs, two viewing systems, and all possible two-way interactions of these effects. This allowed for dependency of the ratings done by the same clinical instructor across both the multiple radiographs and two viewing systems.

Accurate ratings are consistent since they all center on the correct choice. Where ratings are not accurate, they may be consistent (centering around an inaccurate value with little variability) or they may be inconsistent (varying widely). Consistency is thus measured by the standard deviation (SD) of the ratings (square root of the squared difference between the ratings minus the mean of all the ratings provided). To look for differences in consistency, a mixed-model heterogeneous-variance analysis tested for standard deviation differences among the three clinical instructor groups, four correct choice categories, and two viewing systems.


   Results
Top
 Abstract
 Methods
 Results
Discussion
 Conclusion
 References
 
Twenty-four clinical instructors rated bone loss for twenty teeth using a view box. The instructor group consisted of eight dental hygiene faculty members, four graduate students, and twelve periodontal faculty members. The overall response rate was 72.7 percent. All of the dental hygiene faculty and most of the periodontal faculty had ten or more years of clinical experience whereas all but one of the graduate students had less than five years of clinical experience. The upper panel of Table 2Go presents rated bone loss for each correct choice category. For teeth with no bone loss, 74.2 percent (89 of 120) of the clinical instructors’ ratings were accurate. Fifty percent, 52.1 percent, and 94.4 percent of the clinical instructors’ ratings were accurate for categories <15 percent, 15–30 percent, and >30 percent bone loss, respectively. Overall, clinical instructors’ agreement with the correct choice was 70.0 percent. When corrected for chance agreement, this agreement was Kappa=59.1 percent (SE=2.8 percent).


View this table:
[in this window]
[in a new window]
  		Table 2. Accuracy for view box and projection system and agreement between two viewing systems

 
Thirty-five clinical instructors rated bone loss for twenty-five teeth using an LCD projection system. The instructor group consisted of six dental hygiene faculty members, sixteen graduate students, and thirteen periodontal faculty members. The overall response rate was 87.5 percent. There was no change in years of clinical experience for the instructors who viewed radiographs by projection system as compared to those who viewed radiographs by view box. Discrepancies were noted between written and transmitted responses for 1.8 percent of rating; these ratings were omitted from the database. Rated bone loss and comparisons to correct choice for the viewing by LCD projection are shown in the middle panel of Table 2Go. For teeth with no bone loss, 63.3 percent (131 of 207) of the clinical instructors’ ratings were accurate. Fifty-four percent, 48.8 percent, and 94.1 percent of their ratings were accurate for categories <15 percent, 15–30 percent, and >30 percent bone loss, respectively. The overall and chance corrected agreement was 64.5 percent and Kappa=52.7 percent (SE=2.2 percent), respectively.

Eighteen clinical instructors (four dental hygiene, four graduate students, and ten periodontal) provided ratings using both viewing systems. Their ratings were directly compared, and agreement was 76.6 percent (Kappa=67.7 percent, SE=3 percent) (Table 2Go, lower panel). The agreement between the viewing systems is nominally higher than accuracy of either of the two viewing systems (view box=70.0 percent and projected=64.5 percent).

The projection system was found to have a smaller mean difference than the view box (p=0.04). The view box least square (LS) mean difference (LS mean=0.32, SE=0.03, 95 percent CI=0.25–0.39) was significantly greater than the projection system difference (LS mean difference=0.25, SE=0.03, 95 percent CI=0.19–0.31). However, there was some evidence that this difference may not be consistent across the three groups (p=0.07). There was a significant improvement using the projection system only among the graduate students (uncorrected for multiple comparisons p-value=0.01). There was no view box versus projection system difference within the dental hygiene (p>0.90) or periodontal (p>0.50) faculty group.

More than 66.5 percent (886/1333) of all ratings agreed with the correct choice categories, but the differences varied between –2 and +3 (mean=0.22, SD=0.57). The interaction between clinical instructor group and correct choice category was the only interaction that reached statistical significance; however, this interaction was not consistent across the four correct choice categories (p=0.02). In the categories of <15 percent and 15–30 percent bone loss, there was a significant difference among the three clinical instructor groups (Table 3). Periodontal faculty had significantly less error than the dental hygiene faculty (p=0.02), although this difference varied by the correct choice category (p=0.02). Overall, the amount of error varied with the amount of actual bone loss (p<0.0001). Teeth in category <15 percent had significantly larger amounts of average error (LS mean=0.51, 95 percent CI=0.43, 0.60), and teeth in category 15–30 percent had significantly smaller error (LS mean=0.27 (95 percent CI=0.20, 0.34). The teeth with the least amount of error, the smallest differences as compared to the correct choice, were the teeth in category >30 percent bone loss (LS mean=–0.03, 95 percent CI= –0.10, 0.04).

Overall, there was an overestimation of bone loss as indicated by positive mean differences (Table 3, difference column) for categories none, <15 percent, and 15–30 percent bone loss. In the category <15 percent, 49.0 percent of view box ratings and 38.2 percent of projection system ratings were given as 15–30 percent. In the category 15–30 percent, 39.3 percent of view box ratings and 8.1 percent of projection ratings were given as >30 percent. That is, a slightly smaller proportion of clinical instructors overestimated (34.1 percent) bone loss using the projection system, but underestimation doubled to 17.5 percent.

The absolute difference did not vary depending upon the three groups (p=0.24), and there was no evidence for a view box versus projection system difference (p=0.69). However, the absolute difference varied depending on the correct choice (p<0.0001). The correct choice categories <15 percent and 15–30 percent bone loss had the largest absolute differences, but they were not significantly different from one another (approximately 0.5 units).

Standard deviation differences depended upon the three clinical instructor groups, four correct choice categories, and viewing system (LR chi-square=264, df=12, p<.0001). That is, there was some indication that the projection system ratings were less consistent (higher variability) than the view box (chi-square=5.2, p=0.0222). From Table 3, the typical SD of the difference was 0.42 for the projection system and 0.40 for the view box. But the largest differences in consistency were between the four correct choice categories (chi-square=195, df=3, p<.0001). Category >30 percent had about half of the variability (SD {approx} 0.21) of the other correct choice categories. Category none had slightly more consistency (SD {approx} 0.42) than categories <15 and 15–30 percent bone loss (SD {approx} 0.47). There was no difference in consistency between categories <15 and 15–30 percent bone loss. Overall, there was a significant difference between the three rater groups (chi-square=21, df=1, p<.0001). Dental hygiene faculty were the most consistent but inaccurate, and this was most evident in the category >30 percent. The SD for dental hygiene was approximately 0.11, whereas the SD for the graduate students and periodontal faculty was twice that (SD {approx} 0.24). There was no evidence for differences in inconsistency between the three rater groups in the other three correct choice categories. That is, ignoring category >30 percent bone loss, the SD was 0.43, 0.46, and 0.45 for dental hygiene, graduate students, and periodontal faculty, respectively.


   Discussion
Top
 Abstract
 Methods
 Results
 Discussion
Conclusion
 References
 
The focus of this investigation was to determine if there was a difference in accuracy and consistency among clinical instructors’ ratings of percent bone loss using view box and LCD projection system in a dim room. Clinical instructors’ agreement with the correct choice was slightly different for view box and projection system, and rater error varied with the amount of actual bone loss as determined by the Schei ruler. The mean difference was used to demonstrate accuracy as it reflects the net deviation of clinical instructors’ responses from the correct choice. A difference of zero means either the correct choice was always given or equal ratings over and underestimating bone loss were given. The mean difference for the projection system was significantly better than the view box. This was due to the graduate students’ relatively small rater error for categories <15 percent, 15–30 percent, and >30 percent bone loss. The absolute difference was also used to demonstrate accuracy as it reflects any deviation from the correct choice. An increase in the absolute difference indicates a decrease in accurate ratings by clinical instructors. The absolute difference did not vary depending on the three groups or viewing system used. The consistency among clinical instructors was demonstrated by the variability of the difference, the SD around the mean of ratings provided. The projection system ratings were slightly less consistent than view box ratings.

Khocht et al.14 reported examiners rated more sites with bone loss using direct digital radiographs compared to conventional plain film viewing. However, Nair et al.15 and Furkart et al.16 found no differences between examiners’ ratings of percent bone loss using direct digital radiographs compared to conventional plain films. These studies compared examiners’ ratings to surgical measurements or expert consensus of the distance from the CEJ to alveolar bone crest. The "gold standard" used in this investigation was percent bone loss as determined by the Schei ruler.9,16 This technique was chosen since it has been found to be accurate in determining bone loss as compared to surgical measurement and it is efficient and easy to use.17 Study design, examiner experience, training and familiarity with viewing system, and use of direct radiography could have contributed to the differences between our results and the results cited above.15–17 To our knowledge there are no studies that compare radiographs obtained through direct digital radiography to radiographs obtained through scanning plain films. The methods by which these radiographs are processed and projected are different and could affect the radiographs’ resolution, contract, brightness, and magnification. These qualities could in turn influence clinicians’ interpretations of the images. Graduate students had less error using projection system. Since graduate students were, on average, younger than periodontal and dental hygiene faculty, they might be more familiar with viewing projected digitized images because computer technology has been an integral part of their education and training.

To our knowledge, this is the first study to report on consistency among clinicians in determining percent bone loss using two different viewing systems. The differences found in this study could be due to clinicians’ lack of familiarity with the projection system for radiographic viewing. However, digital image preparation,18 magnification,19 and grey-scale manipulation19,20 could have contributed to differences in accuracy and consistency between the two viewing systems. It is important to note that differences between the viewing systems were not consistent across groups, and correct choice categories were confirmed by two authors (SKL and HJT) on both view box and projection system prior to clinical instructors’ viewing.

Overall, overestimation of bone loss was more common than underestimation. Albandar3 reported underestimation of proximal bone level using both conventional and direct digital radiography. Eickholz et al.19 reported that examiners underestimated bone loss by using digitally enhanced radiographs. However, Wolf et al.21 showed examiners overestimated proximal bone loss using direct digital radiography. The gold standard used in these studies was surgical measurement of the distance from the CEJ to the alveolar crest. Our results show a decrease in the overestimation of bone loss with underestimation doubling using the projection system compared to view box. Image magnification,19 grey-scale manipulation,15,20 and use of digitized radiographic images could have contributed to the differences between our results and theirs.

Overall, the amount of error varied with the amount of actual bone loss. A decrease in accuracy and consistency is not unexpected in the categories of <15 percent and 15–30 percent bone loss since a clinical instructor can err on both sides of these middle categories. However, it may be that bone loss of <15 percent and 15–30 percent is more difficult to judge than none or >30 percent. Or it may be that teeth and their corresponding amounts of actual bone loss selected for this study could have contributed to error seen in these two middle categories. In category 15–30 percent, three of the four teeth with actual bone loss of 15 percent were judged inaccurately as <15 percent by up to thirteen clinical instructors, and bone loss for one tooth with actual bone loss of 30 percent was judged inaccurately as >30 percent by up to twenty-five clinical instructors. This pattern was not completely unexpected since actual bone loss was at the border of categories used in this study; however, other observations were less expected. For example, five teeth with 5 percent bone loss were judged inaccurately, and bone loss was overestimated as 15–30 percent by up to nineteen clinical instructors.

Teeth with the same actual bone loss did not always have similar accuracy rates. For example, in category none, tooth #29 had a 97.0 percent accuracy rate yet tooth #19 had a 3.1 percent accuracy rate. Analyses were done to look for significant differences in rating percent bone loss for single versus multirooted teeth, anterior versus posterior, and maxillary versus mandibular; however, none were found. It stands to reason that since teeth are highly variable in their root length, shape, and form, the perceived percent bone loss may be very different from the actual bone loss as measured by the Schei ruler.

There were no differences in rater error between groups for teeth in categories none and >30 percent bone loss; however, in categories <15 percent and 15–30 percent, periodontal faculty had significantly less error than dental hygiene faculty regardless of the viewing system used. Periodontal faculty members have more extensive training and clinical responsibilities than dental hygienists; as diagnosticians, they are more accustomed to assessing and quantifying bone loss as they diagnose periodontal diseases.

Rater error could have arisen from clinical instructors’ rating bone loss from a distance less than or greater than 2 mm apical from the CEJ, failure to recognize anatomical landmarks,22 or inability to judge varying percents of bone loss relative to the tooth’s root length. Radiographic quality, indistinguishable periodontal ligament space, alveolar crest, or root apex, bone density, and trabecular pattern could have also contributed to inaccuracy in radiographic interpretation.

Inaccuracies and inconsistencies among clinical instructors in determining percent bone loss were observed in this study. Clinical instructors’ accuracy was 93.8 percent in rating of percent bone loss for teeth in category >30 percent bone loss, yet accuracy dropped to between 49.0 and 55.1 percent for teeth in categories <15 and 15–30 percent bone loss. Additionally, consistency among clinicians was less for these middle categories than for categories none and >30 percent bone loss. Inaccuracies and inconsistencies among clinicians are ubiquitous problems in both medicine and dentistry.23–35 It is somewhat expected to have inconsistencies among clinical instructors when there are a number of subjective elements that go into making a clinical decision; this may be perfectly acceptable if decisions are based on evidence or accepted practice guidelines. However, determinations of bone loss are based on relationships between anatomical factors, which can actually be measured. Therefore, determining percent bone loss is less subjective than interpretation of other clinical findings that can not be directly measured, and inconsistencies among clinical instructors in this area are less expected and acceptable.

In the undergraduate teaching program, categories of bone loss used in this investigation (none, <15 percent, 15–30 percent, and >30 percent) help establish diagnoses of gingivitis, mild, moderate, and severe periodontitis, respectively. This is not to say that radiographic bone loss is the only factor used to mark the difference between gingivitis or periodontitis or establish the severity of periodontitis. Other clinical findings such as deep periodontal pockets, advanced attachment loss, furcation involvement, and mobility often accompany increased bone loss and need to be considered in determining severity of disease. However, bone loss categories are designed to make clinical instructors and students aware of and sensitive to all diagnostic findings and potential treatment needs. Progression of bone loss—from <15 percent to between 15–30 percent, for example—carries with it the potential need to plan for more complex treatment and/or specialty referral in order to achieve therapeutic success. Furthermore, assessment of radiographic bone loss can be used to determine the results of therapy and need for further treatment. Greenstein and Caton state that "the only subjective ways to monitor PDA (periodontal disease activity) are longitudinal assessments of probing attachment levels and radiographs."36

Inaccuracies and inconsistency of radiographic viewing among clinical instructors may be particularly problematic in a dental school setting where patients are assessed and treated by multiple clinicians. This could lead to errors in establishing diagnoses and prognoses, over- or undertreatment, and increased treatment time and cost. Inaccuracies and inconsistency among clinical instructors may also be problematic in the teaching of radiographic interpretation and determining bone loss. Students may learn to determine bone loss incorrectly or be quite confused by varying ratings among their clinical instructors, making it more difficult to relate radiographic findings to clinical findings and manage patients. In education, students are generally assessed by their ability to generate the "correct answer." Their answer is usually compared to the opinion of the said expert (i.e., the clinical instructor). If clinical instructors’ opinions are constantly changing, then the ability to judge student performance is lost.

Faculty development sessions focusing on review of anatomical landmarks and determining percent bone loss could enhance accuracy and consistency of radiographic interpretation among clinical instructors. This may lead to enhancement of patient management, teaching, student learning, and assessment of radiographic interpretation. Our results show slight differences in accuracy for rating percent bone loss using view box and projection system. Consistency among clinical instructors varied depending on the viewing system used. It is expected that as clinicians become familiar with the projection system for radiographic viewing, inconsistency among them will be comparable for the two systems. Viewing projected digitized radiographs by LCD projector offers several advantages in the educational setting. Many clinicians can view the same radiograph at the same time, making it easier to point out and discuss anatomical landmarks, root length and form, and rationale for rating bone loss. A computer-generated grid could be superimposed over the radiograph indicating the actual category of bone loss. Radiographic quality may be enhanced by computerized image manipulation, and storage of teaching material is less cumbersome with a digitized method. Furthermore, the methods used for processing and projecting digitized radiographic images are readily available at most institutions.

There were limitations with the model used here. Digitized radiographs were scanned using a relatively low resolution and displayed by a fixed-pixel projector. In comparing radiographs processed and displayed using these methods to plain films displayed on a view box, there may be differences in radiographs’ resolution, contrast, and potential for grey-scale manipulation and magnification that could affect image quality. This in turn could impact clinicians’ interpretations of percent bone loss. However, it is important to note that radiographic categories were confirmed independently by two author clinicians (SKL and HJT) prior to clinical instructors’ viewing of radiographs by LCD projector. The overall accuracy rate (agreement with correct choice obtained using the Schei ruler technique on plain films) was 85.0 percent for the periodontal faculty group viewing projected digitized images, which shows accuracy could be obtained by LCD viewing. The image quality of digitized radiographs could have been enhanced by scanning plain films at higher resolution and/or projecting radiographs at higher resolution. However, having clinical instructors view digitized radiographic images using a single high resolution laptop, for example, does not offer the advantages outlined above for group teaching. A better quality radiograph would have been produced by direct digital radiography as compared to the method used for process radiographs used here. However, direct digital radiography is not readily available at all institutions; and obtaining radiographs in this way and via the conventional method so that comparisons could be made between the two radiographic techniques would mean exposing patients to radiation twice. Additionally, Khocht et al. reported that radiographs obtained through the direct digital method and those obtained through the conventional method were not the same in that the former had a higher number of sites with bone loss.14 Other limitations of our investigation include the use of nonstandardized plain films and hence digitized radiographic images involving their exposure, angulations, and composition. One could argue, however, that those characteristics reflect real clinical situations and may not have resulted in a significant limitation. Digital image manipulation and magnification were not standardized for all radiographs. Also, the presence of 71.2 percent of teeth with actual bone loss of 15 percent or 30 percent could have contributed to greater inaccuracies and inconsistencies among instructors seen in the middle two categories. Furthermore, clinical instructors could have discussed the radiographs and rating of percent bone loss with one another throughout the course of the study, and hence "cross-talk" among raters could have influenced results. As further studies are developed, these limitations should be taken into consideration.


   Conclusion
Top
 Abstract
 Methods
 Results
 Discussion
 Conclusion
References
 
Accuracy of clinical instructors’ rating (agreement with correct choice) was slightly different for the view box and the LCD projection system. The mean difference (signed rater error) was lower for the projection system. That is, ratings utilizing the projection system were more accurate than view box ratings. This was due to small rater error among graduate students when using the projection system. The mean absolute difference was not significantly different between the two. Projection system ratings were slightly less consistent than view box ratings. The greatest inaccuracies and inconsistencies were seen in the middle two categories of <15 and 15–30 percent bone loss. The periodontal faculty ratings were the most accurate, while the dental hygiene faculty ratings were the most consistent but the most inaccurate. Overall, overestimation of percent bone loss was more common than underestimation. Accuracy and consistency among clinical instructors are necessary for optimal patient management, assessment of teaching, and student learning. Training sessions using a single projection system for displaying radiographic images is advantageous and may enhance patient care and dental education.


View this table:
[in this window]
[in a new window]
  		Table 3. Mean rater error for each clinical instructor group, correct choice category, and viewing system

 

   Acknowledgments
 
The authors would like to thank Mrs. Barbara Wolfgang and Mrs. Beverly Sutton for their administrative assistance, Drs. Roger Hill and Stephen Soehren for assistance with radiographic selection, and Dr. Donna McClish for consultation on statistical methods. Additionally, the authors would like to acknowledge the periodontal and preventive clinical faculty and graduate students at the University of Michigan, School of Dentistry Department of Periodontics and Oral Medicine for their participation and dedication to this project.


   Footnotes
 
Dr. Lanning is Assistant Professor, Department of Periodontics, Virginia Commonwealth University School of Dentistry and was formerly Clinical Assistant Professor, University of Michigan School of Dentistry; Dr. Best is Associate Professor, Department of Biostatistics, Virginia Commonwealth University; Dr. Temple is Clinical Assistant Professor, Department of Periodontics and Oral Medicine, University of Michigan School of Dentistry; Dr. Richards is Clinical Associate Professor, Department of Periodontics and Oral Medicine, University of Michigan School of Dentistry; Ms. Carey is a Research Assistant, Department of Periodontics and Oral Medicine, University of Michigan School of Dentistry, and Dr. McCauley is Professor and Chair, Department of Periodontics and Oral Medicine, University of Michigan School of Dentistry. Direct correspondence and requests for reprints to Dr. Sharon Lanning at 520 North 11th Street, P.O. Box 980566, Richmond, VA 23298-0566; 804-828-4867 phone; 804-828-0657 fax; sklanning{at}vcu.edu.


   REFERENCES
Top
 Abstract
 Methods
 Results
 Discussion
 Conclusion
 References
 

  1. Carranza FA, Takei HH. Radiographic aids in the diagnosis of periodontal disease. In: Newman MG, Takei HH, Carranza FA, eds. Carranza’s clinical periodontology. Philadelphia: W.B. Saunders Co., 2002:454–68.
  2. Hull PS, Hillman DG, Beal JF. A radiographic study of the prevalence of chronic periodontitis in 14-year-old schoolchildren. J Clin Periodontol 1975;2(4):203–10.[Medline]
  3. Albandar JM. Validity and reliability of alveolar bone level measurements made on dry skulls. J Clin Periodontol 1989;16(9):575–9.[Medline]
  4. Kallestal C, Matsson L. Criteria for assessment of inter-proximal bone loss on bite-wing radiographs in adolescence. J Clin Periodontol 1989;16(5):300–4.[Medline]
  5. Gjermo P, Bellini HT, Santos VP, Martina JG, Ferracyoli JR. Prevalence of bone loss in a group of Brazilian teenagers assessed on bitewing radiographs. J Clin Periodontol 1984;11(2):104–13.[Medline]
  6. Hansen BF, Gjermo P, Bergwitz-Larsen KR. Periodontal bone loss in 15-year-old Norwegians. J Clin Periodontol 1984;11(2):125–31.[Medline]
  7. Hausmann E, Allen K, Clerehugh V. What alveolar crest level on a bite-wing radiograph represents bone loss? J Periodontol 1991;62(9):570–2.[Medline]
  8. Lanning SK, Pelok SD, Williams BC, Richards PS, Sarment DP, Oh TJ, et al. Variation in periodontal diagnosis and treatment planning among clinical instructors. J Dent Educ 2005;69(3):325–37.[Abstract/Free Full Text]
  9. Prichard JF. Interpretation of radiographs on periodontics. Int J Periodontics Restorative Dent 1983;3(1):8–39.
  10. Schei O, Waerhaug J, Lovdal A, Arno A. Alveolar bone loss as related or oral hygiene and age. J Periodontol 1959;30:7–16.
  11. Perry DA, Beemsterboer P, Taggart EJ. Periodontics for the dental hygienist. 2nd ed. Philadelphia: W.B. Saunders, 2001:108–9,180.
  12. Armitage GC. Development of a classification system for periodontal diseases and conditions. Ann Periodontol 1999;4(1):1–6.[Medline]
  13. American Academy of Periodontology. Parameters of care. J Periodontol 2000;71(5):849–83.[Medline]
  14. Khocht A, Janal M, Harasty L, Chang KM. Comparison of direct and conventional intra-oral radiographs in detecting alveolar bone loss. J Am Dent Assoc 2003; 134(11):1468–75.[Abstract/Free Full Text]
  15. Nair MK, Ludlow JB, Tyndall DA, Platin E, Denton G. Periodontitis detection efficacy of film and digital images. Oral Surg Oral Med Oral Pathol Oral Radiol Endod 1998;85(5):608–12.[Medline]
  16. Furkart AJ, Dove SB, McDavid WD, Nummikoski P, Matteson P. Direct digital radiography for the detection of periodontal bone lesions. Oral Surg Oral Med Oral Pathol 1992;74(5):652–60.[Medline]
  17. Bassiouny MA, Grant AA. The accuracy of the Schei ruler: a laboratory investigation. J Periodontol 1975;46(12):748–52.[Medline]
  18. Chen SK, Hollender L. Digitizing of radiographs with a flatbed scanner. J Dent 1995;23(4):205–8.[Medline]
  19. Eickholz P, Riess T, Lenhard M, Hassfeld S, Staehle HJ. Digital radiography of interproximal bone loss: validity of different filters. J Clin Periodontol 1999;26(5):294–300.[Medline]
  20. Guneri P, Lomcali G, Boyacioglu H, Kendir S. The effects on incremental brightness and contrast adjustments on radiographic data: a quantitative study. Dentomaxillofac Radiol 2005;34(1):20–7.[Abstract/Free Full Text]
  21. Wolf B, von Bethlenfalvy E, Hassfeld S, Staehle HJ, Eickholz P. Reliability of assessing interproximal bone loss by digital radiography: intrabony defects. J Clin Periodontol 2001;28:869–78.[Medline]
  22. Lareim TA, Eggen S. Measurements of alveolar bone height at tooth and implant abutments on intraoral radiographs: a comparison of reproducibility of Eggen technique utilization with and without a bite impression. J Clin Periodontol 1982;9(3):184–92.[Medline]
  23. Goldman L, Weinberg M, Weisberg M, Olshen R, Cook EF, Sargent RK, et al. A computer-derived protocol to aid in the diagnosis of emergency room patients with acute chest pain. N Engl J Med 1982;307(10):588–96.[Abstract]
  24. Boom R, Gonzalez C, Fridman L, Ayala JF, Realpe JL, Morales P, et al. Looking for "indicants" in the differential diagnosis of jaundice. Med Decis Making 1986; 6(1):36–41.
  25. Todd BS, Stamper R. Limits to diagnostic accuracy. Med Inform 1993;18(3):255–70.
  26. Mileman PA, Pudell-Lewis D, van der Weele L. Effect of variation in caries diagnosis and degree of caries on treatment decisions by dental teachers using bitewing radiographs. Community Dent Oral Epidemiol 1983; 11(6):356–62.[Medline]
  27. Mileman P, Purdell-Lewis DJ, Dummer P, van der Weele L. Diagnosis and treatment decisions when using bitewing radiographs: a comparison between two dental schools. J Dent 1985;13(2):140–51.[Medline]
  28. Espelid I, Tveit AB, Fjelltveit A. Variations among dentists in radiographic detection of occlusal caries. Caries Res 1994;28(3):169–75.[Medline]
  29. Mileman PA, van der Weele LT. Accuracy in radiographic diagnosis: Dutch practitioners and dental caries. J Dent 1996;18(3):130–6.
  30. Tongsong T, Iamthongin A, Wanapirak C, Piyamongkol W, Sirichotiyakul S, Boonyanurak P, et al. Accuracy of fetal heart-rate variability interpretation by obstetricians using the criteria of the National Institute of Child Health and Human Development compared with computer-aided interpretation. J Obstet Gynaecol Res 2005;31(1):68–71.[Medline]
  31. Balabanova Y, Coker R, Fedorin I, Zakharova S, Plavinskij S, Krukov N, et al. Variability in interpretation of chest radiographs among Russian clinicians and implications for screening programmes: observational study. BMJ 2005;13:379–82.
  32. Moore JH, Goss DL, Baxter RE, DeBerardino TM, Mansfield LT, Fellows DW, et al. Clinical diagnostic accuracy and magnetic resonance imaging of patients referred by physical therapists, orthopaedic surgeons, and nonorthopaedic providers. Ortho Sports Phys Ther 2005;35(2):67–71.
  33. Marbach JJ, Raphael KG, Janal MN, Hirschkorn-Roth R. Reliability of clinical judgments of bruxism. J Oral Rehabil 2003;30(2):113–8.[Medline]
  34. Shetty V, Atchsin K, Belin TR, Wang J. Clinical variability in characterizing mandible fractures. J Oral Maxillofac Surg 2001;59(3):254–61.[Medline]
  35. Persson GR, Schlegel-Bregenzer B, Lang NP, Attstrom R. Education in periodontology: a need for a new teaching model. Eur J Dent Educ 3(2):74–81.
  36. Greenstein G, Caton J. Periodontal disease activity: a critical assessment. J Periodontol 1990;61(9):543–52.[Medline]



This article has been cited by other articles:


Home page
 J Dent EducHome page
S. K. Lanning, A. M. Best, H. J. Temple, P. S. Richards, A. Carey, and L. K. McCauley
Accuracy and Consistency of Radiographic Interpretation Among Clinical Instructors in Conjunction with a Training Program
J Dent Educ., May 1, 2006; 70(5): 545 - 557.
[Abstract] [Full Text] [PDF]

This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Lanning, S. K.
Right arrow Articles by McCauley, L. K.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Lanning, S. K.
Right arrow Articles by McCauley, L. K.

HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS