HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS		QUICK SEARCH:   [advanced] Author: Keyword(s): Year:  Vol:  Page:

This Article Abstract Full Text (PDF) Alert me when this article is cited Alert me if a correction is posted Services Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by MacNeil, R. L. Articles by Neumann, L. M. Search for Related Content PubMed PubMed Citation Articles by MacNeil, R. L. Articles by Neumann, L. M.

Realigning the National Board Dental Examination with Contemporary Dental Education and Practice

Key words: dental education, National Board Dental Examination, dental curriculum, assessment

Submitted for publication 02/06/07; accepted 06/18/07

	Abstract

Top Abstract National boards and dental... Parallel initiatives Suggestions for changes in... Conclusion References

 
Although the National Board Dental Examination (NBDE) was developed for the purpose of supporting the dental licensure process, it can have significant influence on dental school curricula. Efforts to revise and enhance dental curricula, of necessity, must engage stakeholder communities and promote an assessment process that is both valid and relevant to contemporary dental practice. The NBDE uses a systematic and objective process for test development that involves content experts from dental education and practice. This process could be enhanced by reconsidering the types of individuals who should participate in test construction, reconfiguring the groups involved in the process, augmenting training and development of test constructors, and updating the principles and documents that define the competencies and foundational knowledge required for contemporary dental practice. In addition, there is a need for ongoing research and development to explore new testing strategies that stress understanding and the ability to apply information in a problem-solving context. Finally, examinations could be enhanced with a broader array of high-quality supporting case materials through a more structured collaborative arrangement between dental schools and the National Board testing program.

	National Boards and Dental School Curricula

Top Abstract National boards and dental... Parallel initiatives Suggestions for changes in... Conclusion References

 
Although it is clear that the sole purpose of the NBDE is to assist states in determining if new candidates meet minimal knowledge expectations for licensure, the NBDE has a real if unintended impact on the dental curricula of schools whose graduates pursue licensure in the United States.² It appears logical to predict that the content specifications of national examinations (i.e., the array of topics or knowledge areas deemed important enough by the Joint Commission to be allotted space in the examinations) will induce the curriculum committees of dental schools to include and even emphasize these areas in their educational programs. In a similar manner, the methods used within the NBDE to examine candidates in these areas might well influence the examination techniques used in courses.

The hypothesis of this article is that the NBDE influences dental school curricula content and delivery. If this hypothesis is correct, it is then reasonable for dental education institutions to encourage and strive for national examinations that will be contemporary and relevant in terms of both the knowledge base being examined and the method of assessment. Further, it is reasonable to assume that realignment in the examination process to obtain greater relevancy to the contemporary practice of general dentistry would enhance the validity of these examinations for both their intended purpose and as a more broadly applicable measure of student or new graduate competence.

	Parallel Initiatives

Top Abstract National boards and dental... Parallel initiatives Suggestions for changes in... Conclusion References

 
A number of recent initiatives in organized dental education have addressed the question "do the National Boards address current, important, and relevant information?" This was a keynote topic at the American Dental Education Association (ADEA) 47^th Deans’ Conference in November 2005.³ Further, it has been one of the pivotal topics addressed by the ADEA Commission on Change and Innovation in Dental Education (CCI), beginning at its inception in May 2005.⁴ The CCI is an initiative supported by ADEA in conjunction with several key stakeholder organizations in dental education including the American Dental Association (ADA), the Commission on Dental Accreditation (CODA), and the JCNDE. The CCI strives to identify curriculum revision that can maximize the educational process and promote the development of dental graduates optimally prepared for the practice of dentistry over the next decades.⁵ An important initial decision of the CCI was that such change could not be accomplished by dental schools working independently of organizations and groups that impact or influence dental education, including the JCNDE. A central premise of the CCI is that a large group of partners in dental education, including the major change agents like ADEA, CODA, and the JCNDE, must work in close communication and synchrony to successfully achieve change.⁶

Thus, it appears that dental education is at an opportune time to partner with other stakeholders to promote tangible curriculum revision. In the case of student assessment, a desired scenario would be for the JCNDE to understand and recognize the changing concepts emanating nationally and to promote processes whereby national examinations would, at minimum, not deter implementation of such change in dental schools. The converse scenario—one in which national examinations might fail to recognize certain change initiatives while continuing to emphasize traditional or "pre-revision" topics or philosophies in dentistry—could significantly jeopardize the odds of achieving meaningful curriculum change. This article will explore processes that might avert such a situation, while improving examinations independent of whether national change initiatives exist or not.

	Suggestions for Changes in JCNDE Processes to Improve Examinations

Top Abstract National boards and dental... Parallel initiatives Suggestions for changes in... Conclusion References

 
The current examination construction process has been in place for several decades, functions efficiently, and has produced examinations with good psychometric properties.⁷ However, some limited changes, as proposed here, could further improve exam development and, in doing so, better ensure that curriculum change initiatives are appropriately recognized or reflected in national examinations.

Recommendation #1: Carefully consider the formulation of the Test Construction Committees.
It is our opinion that the most important groups or process steps in the construction of JCNDE examinations are the Test Construction Committees (TCCs).¹ While the TCCs cannot change the content specifications for the examination, except for changes in terminology or minor shifts in focus, they do determine the emphasis that will be placed on specific topics falling within each larger content domain. For example, in Operative Dentistry Content Specification 5.2.2: Indirect Restorations, emphasis might be placed on crown or porcelain jacket crown restorations rather than more contemporary CAD-CAM techniques or esthetic veneer techniques. Further, the TCCs determine those test items that will be retained in the item bank and the new test items that will be added to it. In doing so, considerable latitude is afforded to each TCC in creating and selecting test items grounded in specific topics or philosophies of dental science or therapy. While not intended, it is certainly possible that certain topics or philosophies could be stressed while others were overlooked or downplayed. This has the potential to result in a bank of examination questions for a certain content area that might not be as balanced, comprehensive, and contemporary as desired.

Additionally, each TCC has considerable flexibility in determining the type of multiple choice test item used (for example, Type I vs. Paired True-False vs. Cause-and-Effect), although there is a stated JCNDE preference for the simplest format (that being Type I multiple choice completion—a question consisting of a simple stem and one answer option amongst four possibilities).⁸ More importantly, in addition to content emphasis, the TCCs have both flexibility and responsibility for the cognitive level of items, i.e., the extent of critical thinking or problem solving required for responding to an item.

Currently, most TCC members are subject matter experts in their discipline or field and either volunteer for this service or are nominated by their peers. Recently, some leaders in dental education have proposed that TCC members should not be experts but rather general dentists with acknowledged aptitude and expertise in certain subject areas, with the premise that questions emanating from a committee so comprised would tend to be more appropriate for assessing the preparation of candidates for the practice of general dentistry.³ In light of this proposal to enhance participation by general dentists in examination development, we propose an alternative strategy to promote the formulation of examinations that are optimally referenced to the environment of general dental practice in the community.

It is our recommendation that subject matter experts continue to predominantly populate the TCCs but that the following criteria be carefully considered in their appointment: TCC members should a) possess a clear understanding of the competency level (level of knowledge, skills, and values) expected of candidates to practice general dentistry; b) be able to demarcate knowledge levels expected of generalists as compared to specialists, career educators, researchers, or other highly trained professionals in dentistry where there is a legitimately higher expectation of knowledge base; and c) be extremely knowledgeable in their fields while also possessing a keen grasp of and appreciation for ongoing change in the professions of dentistry and dental education. While not a simple task to identify individuals with this combination of traits, we believe that nominating groups, including dental schools, can do a better job in this area. While identifying experts is important, we submit that it is equally important that these experts understand the practice of general dentistry and the level at which a general dentist (versus a specialist or basic scientist) must master scientific principles in order to deliver contemporary, appropriate, and safe dental care.

Recommendation #2: Require test item construction workshops.
Currently, new TCC members have the option to either attend a test item construction workshop or use reference materials provided by the JCNDE to assist them in understanding the "science and art" of formulating questions or test items.^8,9 It is our contention that formal workshops provided by experts in test item construction are critical for understanding and practicing the principles of quality test item construction. Consequently, we propose that all TCC members, upon accepting a TCC assignment, attend a formal JCNDE-sponsored workshop provided by test development experts, including those on the JCNDE staff.

In addition to the routine training of new TCC members, a national symposium and workshop of educators and testing experts should be convened to explore the application of principles of assessment to the development of item types that assess a broader range of complex cognitive behaviors, including comprehension; understanding; predicting; application and interpretation of facts, principles, and methods; analysis; and evaluation. Although there is a perception that multiple choice items are restricted to the measurement of relatively simple learning outcomes, this item type is actually quite versatile and can be useful in measuring a variety of complex learning outcomes.¹⁰ According to Haladyna, the malady of recall and recognition from which most tests suffer is not a function of item format but limited ability to elicit higher levels of thinking in both teaching and testing.¹¹ With adequate training and practice, item writers can successfully write items with higher-level cognitive demand. Azer, for example, provides practical tips to help question writers create multiple choice items that test higher-level thinking and the use of information.¹² Included in his article is a useful graphic model for linking educational objectives (e.g., interpretation of key words in a scenario, problem-solving skills, integration of knowledge) with areas of knowledge (e.g., physiology, biochemistry, pathogenesis) to generate items addressing specific competencies (e.g., understand the significance of key words in a clinical scenario about a patient with diabetes and generate a hypothesis; understand the biochemical changes in the liver and muscles of patients with diabetic ketoacidosis).

Further, the administration of the NBDE in a computerized format presents an opportunity to take advantage of technology’s potential for displaying graphic material, data sets, and other materials for candidate analysis and interpretation. A national symposium would offer an opportunity to review assessment principles and apply relevant dental content to formulate model item types that emphasize critical thinking skills for application in both teaching and testing. In addition to stimulating more innovative use of multiple choice items, a symposium could also encourage the development or application of new item types, such as extended matching or the key feature item type recently adopted for the National Board Medical Examinations,¹³ and provide more experience with the testlet format recently adopted for the NBDE Part I.

Recommendation #3: Establish independent committees for examination construction design.
As stated above, test items are currently selected and formulated into complete examinations by JCNDE staff, with help from a small number of consultants with prior experience on the TCCs. We believe that the consultant review step in examination formulation is imperative and should occur for all examinations. Further, we recommend a change in the current consultant review approach such that each NBDE examination would have a dedicated Examination Formulation Committee comprised of four or five members. This select group would be nominated by major stakeholder organizations. Extreme care would be taken to identify individuals who have an optimal grasp of the demands and expectations of general dentistry along with the scientific foundation required for contemporary dental practice. These individuals need not be experts in any particular field of dentistry and indeed it may be preferable that they not be experts. While only a limited number of individuals may exist with these characteristics, it should be possible to populate a small, select committee from candidates from across the nation. Ideally, this committee should be formulated to achieve a balance among the concerns, desires, and aspirations of the following constituencies: dental academia (ADEA), examination and licensing bodies (American Association of Dental Examiners and state boards), and the practice community (ADA).

The sole purpose of the Examination Formulation Committees would be to construct "relevant and realistic" examinations using test item banks populated by items generated by the TCCs. The committee would not be permitted to change the test items provided by the TCC content experts or the test specifications. It could, however, do the following:

• Screen and select test items deemed appropriate for the assessment of candidates for competency in general dentistry.
  
• Determine the distribution of test items across the content specification matrix so that an appropriately broad array of topic areas important to contemporary dental practice is presented. This would reduce the possibility of an examination possessing an internal bias or a skewed emphasis on certain topics or philosophies.
  
• Approve the final version of each examination for distribution to candidates.
  
• Provide feedback to the TCCs and the JCNDE on the need for test items in certain areas or levels of understanding and on the quality and appropriateness of case materials. Ask the TCCs to provide additional test items to diversify or augment the test item bank.
  
• Continually refine the test item bank by deleting items of inappropriate difficulty or emphasis while endorsing test items deemed of good quality for the national examination.
  
• Respond to JCNDE requests as they arise.
  

The format described is employed in part by the National Dental Examining Board of Canada in creation of its national examinations and has proved very successful.¹⁴ The approach has the distinct benefit of allowing the TCCs to concentrate on what they do best (i.e., create test items based on subject matter expertise), while permitting an independent committee (e.g., the Examination Formulation Committee) with a contemporary perspective on general dentistry to use these test items in design of a realistic and relevant examination oriented toward the practice of general dentistry. This approach could greatly aid in reducing the undesired possibility of examinations that may concentrate on outdated principles or concepts that are distant from those deemed critical and central to the development of competency in general dentistry.

Recommendation #4: Ensure that foundational and guiding documents used in dental education and candidate assessment truly describe and reflect the contemporary practice of general dentistry.
ADEA should continue its current initiative, under the oversight of the CCI, to define the new Competencies for the General Dentist. It is highly probable that the 1997 ADEA competencies document needs revision to keep pace with ongoing changes in dental practice and dental education, and we applaud the process by which a new document is currently being developed to reflect the current environment. This new document can then be used in future practice analysis surveys to ensure that the content specifications for future National Board examinations are as current and relevant as possible.

Recommendation #5: Research, develop, and pilot new examination and item formats.
The JCNDE should continue its current Innovative Dental Assessment Research and Development (IDEA) Small Grants Program to assist organizations, including dental schools, in developing new examinations and testing strategies for students and licensure candidates.¹⁵ An increase in funding for projects within this program would encourage more faculty and schools to apply and would better support test instrument development. Further, dental schools and their faculty should be encouraged to become more involved in this area to not only improve examination approaches in their home institutions but to share novel or promising approaches with other schools and the JCNDE. The JCNDE has already shown willingness to pilot new examination formats (e.g., the testlet format for Part I), and it should continue to be innovative and forward-looking in this vein. Further, the JCNDE has stated that it desires to create examinations that stress candidate understanding and the ability to apply information in a problem-solving context. To do this, it appears important to migrate away from the traditional item types that focus on recall of descriptive, factual information where candidate success is probably more dependent on rote memorization than an ability to understand and think critically. A close partnership between the JCNDE and dental schools, where a wealth of educational expertise exists, would hasten the development of examinations that target the ability of candidates to synthesize information, think critically, and truly demonstrate readiness for dental practice.

Recommendation #6: Dental schools should enter into an educational collaborative with the JCNDE.
As the NBDE examinations move toward more clinically referenced exam formats, it will be essential to develop a library of contemporary case-based materials (e.g., clinical photographs, radiographs, and patient data sets) that are of excellent quality and appropriate scope. The current process of requesting TCC or ADA members to volunteer case-based materials for examinations will not accomplish this goal. We recommend that dental schools enter into a collaborative agreement with the JCNDE to provide such materials. Almost all postdoctoral advanced programs extensively document the oral condition and findings of patients under care, and many pre-doctoral programs do as well. The contribution of a single case by each dental school on an annual basis would provide a rich resource that could substantially expand the scope and depth of the NBDE item banks and quality of the assessment process. In exchange, schools might receive compensatory services from the JCNDE such as test item construction workshops for faculty, practice examinations, etc.

	Conclusion

Top Abstract National boards and dental... Parallel initiatives Suggestions for changes in... Conclusion References

 
Overall, we submit that the NBDE examinations are of good quality and capably accomplish the task of evaluating candidates for licensure. This assessment is validated by evidence from the technical report of the JCNDE¹⁶ and by the continued use of the exams by all licensing jurisdictions. The educators, practitioners, and other individuals who volunteer their time in JCNDE activities, along with commission staff, do an admirable job and should be commended for their commitment and service. The six recommendations provided here build on that solid base and aim to strengthen the examinations, especially as dental education morphs in response to changes in the profession, science, and society. We believe that these recommendations are timely and appropriate and, importantly, are achievable in that they can be implemented without major disruption to current processes and with limited new expense.

	Footnotes

 
Dr. MacNeil is Dean, School of Dental Medicine, University of Connecticut and a past Chair of the Joint Commission on National Dental Examinations; and Dr. Neumann is Senior Vice President, Education/Professional Affairs, American Dental Association. Direct correspondence and requests for reprints to Dr. Laura M. Neumann, American Dental Association, 211 E. Chicago Avenue, Chicago, IL 60611; 312-440-2712 phone; 312-587-4105 fax; neumannL{at}ada.org.

	References

Top Abstract National boards and dental... Parallel initiatives Suggestions for changes in... Conclusion References

 

1. Neumann LM, MacNeil RL. Revisiting the National Board Dental Examination. J Dent Educ 2007; 71(10):1281–92.[Abstract/Free Full Text]
2. Kassebaum DK, Hendricson WD, Taft T, Haden NK. The dental curriculum at North American dental institutions in 2002–03: a survey of current structure, recent innovations, and planned changes. J Dent Educ 2004; 68(9):914–31.[Abstract/Free Full Text]
3. American Dental Education Association. 47th Deans’ Conference, November 2005, Ft. Lauderdale, FL.
4. What is the ADEA Commission on Change and Innovation in Dental Education? How did it get started? Bulletin of Dental Education, December 2005.
5. Pyle M, Andrieu SC, Chadwick DG, Chmar JE, Cole JR, George MC, et al. The case for change in dental education. J Dent Educ 2006; 70(9):921–4.[Abstract/Free Full Text]
6. Kalkwarf KL, Haden NK, Valachovic RW. ADEA Commission on Change and Innovation in Dental Education. J Dent Educ 2005; 69(10):1085–7.[Free Full Text]
7. Cole JR. Legal status of dentistry and licensure. J Am Coll Dent 2002; 69(2):6–12.[Medline]
8. Joint Commission on National Dental Examinations. Test item development guide. Chicago: Joint Commission on National Dental Examinations, 2007.
9. Joint Commission on National Dental Examinations. Orientation manual for National Board dental and dental hygiene test constructors. Chicago: Joint Commission on National Dental Examinations, 2007.
10. Gronlund NE, Linn RL. Measurement and evaluation in teaching. New York: MacMillan, 1990.
11. Haladyna TM. Developing and validating multiple-choice test items. 3rd ed. Mahwah, NJ: Lawrence Erlbaum Associates, 2004.
12. Azer SA. Assessment in a problem-based learning course. Biochem Mol Biol Educ 2003; 31:428–34.
13. Page G, Bordage G, Allen T. Developing key-feature problems and examinations to assess clinical decision-making skills. Acad Med 1995; 70(3):194–201.[Medline]
14. Gerrow J. Personal communication, June 5, 2006.
15. Joint Commission on National Dental Examinations. Innovative dental assessment research and development grants. Chicago: Joint Commission on National Dental Examinations, 2006.
16. Joint Commission on National Dental Examinations. Technical report. Chicago: Joint Commission on National Dental Examinations, 2006.

This article has been cited by other articles:

				H. F. Thomas What the ADEA CCI Series of Articles Means to Me: Reflections of a Dental School Dean J Dent Educ., February 1, 2009; 73(2): 172 - 176. [Abstract] [Full Text] [PDF]

				J. E.N. Albino, S. K. Young, L. M. Neumann, G. A. Kramer, S. C. Andrieu, L. Henson, B. Horn, and W. D. Hendricson Assessing Dental Students' Competence: Best Practice Recommendations in the Performance Assessment Literature and Investigation of Current Practices in Predoctoral Dental Education J Dent Educ., December 1, 2008; 72(12): 1405 - 1435. [Abstract] [Full Text] [PDF]

This Article Abstract Full Text (PDF) Alert me when this article is cited Alert me if a correction is posted Services Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by MacNeil, R. L. Articles by Neumann, L. M. Search for Related Content PubMed PubMed Citation Articles by MacNeil, R. L. Articles by Neumann, L. M.