HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS		QUICK SEARCH:   [advanced] Author: Keyword(s): Year:  Vol:  Page:

This Article Abstract Full Text (PDF) Alert me when this article is cited Alert me if a correction is posted Services Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Stewart, C. M. Articles by Smith, G. E. Search for Related Content PubMed PubMed Citation Articles by Stewart, C. M. Articles by Smith, G. E.

Relationship Between Performance in Dental School and Performance on a Dental Licensure Examination: An Eight-Year Study

Key words: dental licensure exam, exam performance predictors, academic ranking

Submitted for publication 03/14/05; accepted 04/25/05

	Abstract

Top Author information Abstract Methods Results Discussion Conclusion References

 
This study assessed relationships between academic performance in dental school and "first attempt" performance on a state dental licensure examination for 1996–2003 graduates from the University of Florida College of Dentistry (UFCD). The 524 graduates were ranked into quartiles based on graduating GPA. Using analysis of variance (ANOVA), the students’ mean exam score (or exam section score) for each respective quartile (n=131) was compared with mean score for graduates in the combined four quartiles (n=524). ANOVA assessments, by quartile, were performed for the following six measures: 1) overall composite score on the dental licensure exam, 2) clinical periodontics section, 3) clinical amalgam section, 4) combination of clinical periodontics and clinical amalgam, 5) laboratory (manikin exam) with a written prosthodontic exam, and 6) manikin exam without the prosthodontic exam. For the overall exam and all exam sections, a significant (p<0.001) relationship was found between higher mean exam scores and academic ranking in quartile 1. A significant relationship was found between performance (lower mean scores) and ranking in quartile 4 for all exam sections, with the exception of the clinical periodontal section. The results of this study indicate a correlation between performance in dental school and performance on the Florida dental licensure exam for 1996–2003 UFCD graduates.

One might anticipate a positive relationship between dental school performance and successful first attempts on clinical licensure exams.¹² However, current reports in the dental literature yield inconclusive results. A recent eight-year retrospective analysis involving University of Florida College of Dentistry graduates and the Florida dental licensure exam found no significant relationship between senior mock board performance and numbers of clinical procedures and successful performance on the licensure exam, but did find a significant relationship between the clinical Class II amalgam procedure on the mock board and subsequent licensure exams.¹⁰ A report looking at a single year’s results from an examination given by the North East Regional Board (NERB) could not confirm a positive relationship between the school performance measures (clinical procedures, fourth-year class rank, and GPA) and the NERB outcomes of passing the clinical restorative procedures (Class II amalgam and Class III or IV composite) on the first attempt. In addition, a wide distribution of class ranks was found in groups that both failed and passed the NERB examination. However, a positive correlation was observed between academic performance and those who passed the manikin (SIM) portion of the NERB.⁸

A more recent report indicated that, over a nine-year period, there was no significant difference in class rank percentile between those who passed the RESTOR section (restorative clinical exercise) of the NERB examination and those who failed it.¹¹ The power of this report was that it used data from a nine-year period. Its authors concluded that future studies would be helpful to determine if trends could be generalized to other schools and testing agencies.

To help elucidate the relationship between performance in dental school and performance on a dental licensure examination, this study was conducted using scores from 1996–2003 University of Florida College of Dentistry (UFCD) graduates’ "first attempt" on the Florida dental licensure examination and their academic performance measured by graduating GPA.

	Methods

Top Author information Abstract Methods Results Discussion Conclusion References

 
The University of Florida Institution Review Board granted approval for this project. The study population consisted of UFCD graduates from 1996 through 2003 who attempted the Florida dental licensure exam immediately upon graduation. The total number of subjects (n=524) included 356 males and 168 females.

The composition of the Florida dental licensure exam did not experience a major design revision during the eight years of this analysis. A revision of the manikin portion was implemented beginning with the June 2000 exam. A Class II composite restoration was added, and a pin-amalgam preparation and restoration were deleted. In addition, a three-unit bridge preparation was added, and the single-unit fixed abutment preparation was deleted. The selected clinical procedures and subject content for the written portions have remained constant over time. The patient-based clinical procedures of the state licensure exam consist of a Class II amalgam preparation and restoration, which comprises 25 percent of the exam, and clinical periodontal scaling and root planing on five teeth, which comprises 15 percent of the total licensure exam. A simulated or laboratory section constitutes 60 percent, which includes five dentoform procedures and a written prosthodontic exam. The five manikin procedures include a fixed bridge preparation (20 percent), a lass II composite restoration on a preprepared tooth (10 percent), a Class II amalgam restoration on a preprepared tooth (5 percent), a Class IV composite restoration (5 percent), an endodontic access and obturation on an extracted natural anterior tooth, a premolar, cuspid, or incisor (10 percent), and a written prosthodontic examination (10 percent). Scores range from 0 to 5 on each procedure. To receive Florida dental licensure, the candidate must achieve a composite grade of 3.0 or higher on these procedures, plus pass the written examination on Florida Laws and Rules with a score of 75 percent or higher.

For purposes of assessment, dental school performance was compared with performance on the entire exam and on the clinical and laboratory sections. More specifically, examination performance was grouped into six sections as follows: 1) overall state licensure board examination score, 2) clinical periodontics section, 3) clinical Class II amalgam, 4) combined score on Class II amalgam and periodontics, 5) laboratory (manikin exam) with the prosthodontic written exam, and 6) manikin exam without the prosthodontic written exam. Subsections #2–#6 consisted only of candidate performance on that specific section. A significant relationship was assumed at p<0.001.

Graduates from each year were placed in quartiles, based on graduating GPA. For example, the top 25 percent of each graduating class from 1996 to 2003 comprised quartile 1 for this analysis. The bottom 25 percent of each graduating class was combined to comprise quartile 4. Quartiles 2 and 3 were developed in a like manner. The initial quartile ranking was done for each year to avoid possible influence from grade inflation during the eight years. Each quartile contained 131 students. Using analysis of variance (ANOVA), the students’ mean exam score (or exam section score) from each respective quartile was compared with the mean score for graduates in the combined quartiles (n=524). To assess whether changes implemented in June 2000 would impact the composite ANOVA results, analysis for each separate year was conducted. While N per quartile was low, no significant differences (p<0.01) were detected.

To further characterize the relationships, a second analysis was conducted to determine the percent of students who failed the exam (or exam section) in each academic quartile. The intent of the second analysis was to enhance the understanding of relationships between performance in dental school and performance on the dental licensure examination.

	Results

Top Author information Abstract Methods Results Discussion Conclusion References

 
Tables 1–6 provide two analyses for each licensure exam section. The first analysis is results of ANOVA by quartile for each of the six licensure exam groupings. The second is the performance distribution by quartile. For each exam section, the number and percent of graduates who passed and failed the exam or exam section are provided by quartile.

Tables 2, 3, and 4 demonstrate relationships for the clinical portions of the licensure exam. Table 2 shows relationships for performance on the clinical periodontic portion of the exam. Of the 524 graduates, 79.6 percent passed the clinical periodontic section. A significant (p<0.001) relationship was detected between the mean performance of graduates in quartile 1 vs. all graduates. The mean score, 3.64 for quartile 1, was significantly higher than the mean score for the combined four quartiles: 3.36. There was no significant relationship between the mean score of candidates in quartiles 2, 3, or 4 when compared with mean score of all graduates. Looking at quartile distribution and performance, 13.7 percent from quartile 1 failed this section of the exam. Sixteen percent from quartile 2 failed this section, 26.7 percent from quartile 3, and 25.2 percent from quartile 4 failed the periodontic section.

Table 3 shows relationships for performance on the clinical Class II amalgam procedure. Of 524 graduates, 75.4 percent passed this section. Significant relationships were found between quartile 1 and quartile 4 and all graduates. The mean score for graduates in quartile 1 was 3.60, which was significantly higher than the mean score for all quartiles: 3.34. In contrast, the mean score for those in quartile 4 was 3.05, significantly lower than the mean of all quartiles. No significant relationships were detected for quartiles 2 and 3. The quartile performance distribution indicated that the fewest failures occurred in quartile one: 12.2 percent. With each succeeding quartile, the number of failures increased. This is consistent with findings in Table 1 related to the exam in its entirety, but not consistent with the performance on the periodontal section. Table 4 demonstrates relationships between performance on the combined clinical procedures, the Class II amalgam and periodontics procedures. Of the total number of graduates, 75.0 percent passed and 25.0 percent failed this section. This result was very similar to the pass rate for the clinical amalgam section. Consistent with trends noted in Tables 1 and 3, significant relationships were found between quartile 1 and quartile 4 and mean scores for combined quartiles. The mean score for those in quartile 1 (3.61) was significantly higher than the mean score for combined quartiles: 3.35. The percent of failures increased as the class rank decreased. The failure rate from Q2 was 13.7 percent, from Q3 was 33.6 percent, and from Q4 was 39.7 percent.

Table 5 provides relationships between performance on the simulated section, consisting of the manikin exam and written prosthodontic exam. The overall pass rate for this section was 79.8 percent. ANOVA detected significant relationships between Q1 and Q4 and the mean scores for combined quartiles. The mean score for those in quartile 1 was 3.56, significantly higher than the mean score for combined quartiles: 3.36. The mean score for quartile 4 was 3.22, significantly lower than the combined quartiles. Mean scores for Q2 and Q3 demonstrated no significant relationships with the mean score for the combined quartiles. Quartile 1 contained the lowest percent of failures, 8.4 percent. As the class rank decreased, the number of failures increased. The failure rate for Q2 was 14.5 percent, for Q3 was 25.2 percent, and for Q4 32.8 percent.

Table 6 demonstrates relationships for performance on only the manikin section. Of the total subject pool, 67.2 percent passed and 32.8 percent failed this section. This section demonstrated the highest failure rate of all exam sections. Significant relationships were detected between the mean scores of Q1 and Q4 and mean scores of combined quartiles. The mean score in Q1 (3.41) was significantly higher than the mean of combined quartiles (3.20). The mean score of Q4 (3.03) was significantly lower than the mean of combined quartiles. The quartile distribution for those who failed this section was 17.6 percent from Q1, 29 percent from Q2, 38.2 percent from Q3, and 46.6 percent from Q4.

	Discussion

Top Author information Abstract Methods Results Discussion Conclusion References

 
A dental educator and a dental board examiner would hope that an independent licensure exam would detect those who cannot perform at a level of minimal competence, but would not "fail" those who are competent. In contrast with results reported by Ranney et al.,¹¹ the academic performance of UFCD graduates from 1996 to 2003 correlated with performance on the Florida state dental licensure examination. Results of this study indicated a significant relationship between quartile ranking, determined by graduating GPA, and overall performance on the Florida examination.

We recognize that dental licensure in Florida is determined by a composite performance score on the entire exam. Accordingly, the analysis of overall exam performance, as depicted in Table 1, represents the most meaningful relationships. However, for purposes of performance comparison, exam subsections were analyzed as independent sections.

For all six analyses, the least number of failures were in quartile 1. Of the seventy-five graduates who failed this exam during the previous eight years, only five students (6.7 percent) were in quartile 1. The highest percent of candidate failures (45.3 percent) were in quartile 4. For the overall exam performance, and four of the five exam sections, failures increased as academic performance decreased. Significant relationships were found between performance in dental school and performance on the overall exam for quartile 1 and quartile 4. This same significant relationship was also observed for all sections of the exam, with the exception of clinical periodontics. While a significant relationship was identified between quartile 1 academic performance and passing the clinical periodontics section, relationships for quartiles 2, 3, and 4 were not significant. We might hypothesize that this lack of correlation with GPAs could be due to lack of examiner calibration or lack of understanding by candidates regarding appropriate patient selection and evaluation criteria. Because the state board examiner standardization procedures and inter- and intra-examiner reliability data were not available, these questions could not be addressed.

Consideration was given to assessing the laboratory exam section with and without the prosthodontic written exam because the majority of graduates at UFCD pass the written prosthodontic exam. Analyzing these sections separately revealed expected results. Considering the manikin exam and written prosthodontic exam together resulted in a 20.2 percent section failure rate. Evaluating the manikin exam alone revealed a 32.8 percent failure rate. Consequently, the manikin portion could be interpreted as the most challenging portion, based on the highest percent of failures.

It is interesting to speculate that if the licensure exam had only consisted of the combined clinical Class II amalgam and periodontal procedures, the failure rate would have been 25 percent, based on this analysis. If one considered only the manikin procedures, the failure rate would have been even greater: 32.8 percent. The failure rates on the clinical portion and the manikin portion are higher than the overall exam failure rate of 14.3 percent. What raises the percent of passing candidates is the written prosthodontic exam worth 10 percent and the method of composite grading. Candidates can fail procedures and sections and still pass the overall exam with an overall performance score of 3.0 or greater.

The manikin portion of the laboratory section accounts for 50 percent of the entire licensure exam. Recognizing that nearly 33 percent of the UFCD graduates failed this section will provide the faculty with an opportunity to improve graduates’ performance through additional instruction and mentored practice sessions.

Any attempt to explain differences between results reported by Ranney et al.¹¹ and this analysis can only be speculative. Performance on the RESTOR portion of the NERB exam did not correlate with class rank, while performance on the clinical Class II amalgam and periodontal portions of the Florida exam did correlate with graduating GPA. Ranney et al. suggested that a lack of correlation was related to exam validity, based on variation in certain failure rates year to year, inconsistency between the results of the manikin portion and the patient restorative portion, and low interexamination reliability (year to year). According to Figure 1 in the report by Ranney et al., the failure rates for the nine-year analysis appeared to range from 15 percent to 60 percent. During the eight years of the Florida exam analysis, the UFCD "first attempt" failure rates ranged from 3 percent to 27 percent.

UFCD has a positive working relationship with the Florida State Board of Dentistry based on mutual respect, effective communication, and understanding and acknowledgment of the roles and responsibilities performed by each group. Through a contemporary predoctoral curriculum, the college seeks to graduate a competent dentist with the skills, knowledge, and values to begin independent practice. One of the primary missions of the board is to protect the health and well-being of the public, with one of those responsibilities being granting dental licensure. We believe that this relationship, coupled with an effective curriculum, which includes a rigorous senior mock board examination, may be key factors in finding a correlation between performance in dental school and performance on the state licensure exam.

	Conclusion

Top Author information Abstract Methods Results Discussion Conclusion References

 
Significant correlations were found between dental school performance for 1996–2003 UFCD graduates and performance on the Florida dental licensure examination. Significant relationships were noted between performance in dental school, based on quartile ranking according to graduating GPA, and overall performance on the Florida dental licensure exam, performance on the clinical Class II amalgam procedure, performance on the combined clinical Class II amalgam and periodontal procedures, and performance on the laboratory section, with and without the written prosthodontic exam. Significant (p<0.001) relationships were noted between exam performance and academic performance for quartiles 1 and 4 for these sections. For the periodontics section, a significant relationship was detected only for quartile 1.

An analysis of failures on exam sections resulted in discovery that the manikin portion was associated with the highest failure rate. These results will provide opportunities for UFCD to improve licensure exam performance of future graduates through additional instruction and practice on the manikin procedures.

As states and regions contemplate future exam models, a retrospective analysis of exam performance with academic performance might be helpful. While it is hoped that no student who has successfully completed a dental curriculum would fail a licensure exam, it does occur. Validity of a dental licensure examination might be supported, in part, by correlation between candidate academic performance during dental school and performance on the licensure examination.

	Author Information

Top Author information Abstract Methods Results Discussion Conclusion References

 
Dr. Stewart is Associate Professor and Director, Division of Oral Diagnostic Sciences, Department of Oral and Maxillofacial Surgery and Diagnostic Sciences; Dr. Bates is Associate Professor and Senior Associate Dean of Operations and Clinical Affairs, Department of Prosthodontics; and Dr. Smith is Professor, Department of Operative Dentistry—all at the University of Florida College of Dentistry. Direct correspondence and requests for reprints to Dr. Carol M. Stewart, Division of Oral Diagnostic Sciences, Department of Oral and Maxillofacial Surgery and Diagnostic Sciences, P.O. Box 100414, University of Florida College of Dentistry, Gainesville, FL 32610-0414; 352-392-2502 phone; 352-392-2507 fax; cstewart{at}dental.ufl.edu.

	REFERENCES

Top Author information Abstract Methods Results Discussion Conclusion References

 

1. Buchanan R. Problems related to the use of human subjects in clinical evaluation/responsibility of follow-up care. J Dent Educ 1991;55:797–801.[Medline]
2. Formicola AF, Shub JL, Murphy FJ. Banning live patients as test subjects on licensing examinations. J Dent Educ 2002;66:605–9.[Abstract]
3. Hasegawa TK Jr. Ethical issues of performing invasive irreversible dental treatment for purposes of licensure. J Am Coll Dent 2002;69(2):43–6.[Medline]
4. Kalkwarf K. Commentary on Melnick, Pattalochi, and Formicola papers. J Dent Educ 2002;66:610–1.
5. Pattalochi RE. Patients on clinical board examinations: an examiner’s perspective. J Dent Educ 2002;66:600–4.[Medline]
6. Dugoni AA. Licensure: entry level examinations-strategies for the future. J Dent Educ 1992;56:251–3.[Medline]
7. Damiano PC, Shugars DA, Freed JR. Clinical board examinations: variation found in pass rates. J Am Dent Assoc 1992;123:68–73.[Abstract]
8. Ranney RR, Wood M, Gunsolley JC. Works in progress: a comparison of dental school experiences between passing and failing NERB candidates. J Dent Educ 2003;67: 311–6.[Abstract]
9. Chambers DW. Portfolios for determining initial licensure competency. J Am Dent Assoc 2004;135:173–84.[Abstract/Free Full Text]
10. Stewart CM, Bates RE, Smith GE. Does performance on a school-administered mock board predict performance on a dental licensure exam? J Dent Educ 2004;68(4):426–32.[Abstract]
11. Ranney RR, Gunsolley JC, Miller LS, Wood M. The relationship between performance in a dental school and performance on a clinical examination for licensure: a nine-year study. J Am Dent Assoc 2004;135:1146–53.[Abstract/Free Full Text]
12. Hall EH. Steering committee/educators’ meeting: comments from the dental discussion groups. Washington, DC: North East Regional Board of Dental Examiners, 2001:6.

This article has been cited by other articles:

				J. E.N. Albino, S. K. Young, L. M. Neumann, G. A. Kramer, S. C. Andrieu, L. Henson, B. Horn, and W. D. Hendricson Assessing Dental Students' Competence: Best Practice Recommendations in the Performance Assessment Literature and Investigation of Current Practices in Predoctoral Dental Education J Dent Educ., December 1, 2008; 72(12): 1405 - 1435. [Abstract] [Full Text] [PDF]

				D. C. Holmes, J. V. Doering, and M. Spector Associations Among Predental Credentials and Measures of Dental School Achievement J Dent Educ., February 1, 2008; 72(2): 142 - 152. [Abstract] [Full Text] [PDF]

				J. D. Gerrow, H. J. Murphy, M. A. Boyd, and D. A. Scott An analysis of the contribution of a patient-based component to a clinical licensure examination. J Am Dent Assoc, October 1, 2006; 137(10): 1434 - 1439. [Abstract] [Full Text] [PDF]

				D. Hammond, C. W. Buckendahl, R. R. Ranney, and R. Hambleton Do portfolio assessments have a place in dental licensure? J Am Dent Assoc, January 1, 2006; 137(1): 30 - 41. [Full Text] [PDF]

This Article Abstract Full Text (PDF) Alert me when this article is cited Alert me if a correction is posted Services Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Stewart, C. M. Articles by Smith, G. E. Search for Related Content PubMed PubMed Citation Articles by Stewart, C. M. Articles by Smith, G. E.