| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
Perspectives |
Key words: assessment, competence, dental education, dental students
Preface from the Chair of the ADEA CCI:
In 2005, the American Dental Education Association established the Commission on Change and Innovation in Dental Education (ADEA CCI) to build a consensus within the dental education community about innovative changes that are necessary in the education of general dentists to ensure that dental school graduates enter the profession fully competent to meet the oral health needs of the public.1,2 As a part of its work, the ADEA CCI established several task forces, including the Task Force on Student Outcomes Assessment. The specific goal of this task force is to improve assessment practices in dental education. To this end, the task force, led by Dr. Gene A. Kramer, director of testing services at the American Dental Association, has created a "toolbox" designed to provide dental educators with a variety of techniques and methods for assessing the acquisition by students of competencies associated with the successful practice of dentistry. The assessment tools described range from familiar and frequently used assessment techniques such as multiple-choice and short answer essay items to newer and less familiar methods such as objective structured clinical examinations (OSCEs), portfolios, and triple jump exercises.
In the survey on competency assessment strategies used by dental schools that was reported in the December 2008 issue of the Journal of Dental Education,3 it was found that the multiple-choice format is used most often by dental educators in assessing most student competencies, despite evidence to suggest that some competencies might be better assessed with other tools. The task force hopes that, by describing a range of tools and their possible applications, educators will explore ways to expand how they assess the knowledge, skills, and abilities of their students as they relate to the competencies necessary to function as beginning general dentists.
This Dental Student Assessment Toolbox is a first step in introducing alternative assessment tools and strategies into pre-doctoral dental education. The content of the toolbox will be honed and expanded in coming years to continue to meet the evolving needs of the dental educational community.
Stephen K. Young, D.D.S.
Chair, ADEA Commission on Change and Innovation in Dental Education
Chair, ADEA CCI Task Force on Student Outcomes Assessment
Assessment is an essential component of the educational experience. Assessment of students’ progress in the dental curriculum ensures that they are acquiring the necessary knowledge, procedural/technical skills, problem-solving capacities, and critical thinking abilities. There are a variety of purposes to conducting assessment in the educational environment. The outcomes of assessment can be used to diagnose student strengths and weaknesses, to identify potential programmatic or curricular challenges, and to monitor students’ progression toward, and ultimate attainment of, designated competencies that comprise the capacities of entry-level practitioners.
Assessment, however, is not a unitary concept when it comes to methodology. There are any number of methods that can be used, depending on the competencies being assessed. Using only one or two methods to assess students’ attainment of the wide variety of knowledge, skills, and abilities supporting dental competencies would not be efficient or effective. Assessment methods range from simple written formats with relatively low levels of fidelity to actual demonstrations of capacity to perform skills in high fidelity situations that approximate the circumstances of general dental practice in the community. This Dental Student Assessment Toolbox was created to assist dental educators with the critical yet challenging task of determining the optimal methods for assessing students’ progression toward and ultimate attainment of the competencies designated as necessary for the entry-level practice of general dentistry.
 |
Glossary |
|---|
 TopAuthor information Glossary Table and Descriptions of...Written assessments: selected...SourcesAssessment by observationSourcesMultisource assessmentSourcesSimulationSourcesMulti-competency, comprehensive...SourcesWork samplesSourcesGeneral references on assessmentAppendixReferences |
|---|
Assessment and Assessment Tools:
In the broad sense, assessment involves the gathering of information to determine the knowledge, skills, abilities, and performance levels of students or candidates for graduation, licensure, or certification. Assessment tools comprise a wide range of instruments and methodologies designed to gather this information for feedback, diagnostic purposes, and identifying successful attainment of competence.
Competency:
A complex behavior or ability essential for the general dentist to begin independent, unsupervised dental practice; it assumes that all behaviors and skills are performed with a degree of quality consistent with patient well-being and that the general dentist can self-evaluate treatment effectiveness.
Fidelity:
Fidelity refers to the similarity of the assessment tool to the actual competency or student performance being assessed. A high fidelity tool is one that is very similar to the actual performance.
Formative Assessment:
Formative assessment involves the accumulation of evaluative information for diagnostic purposes and, in the educational context, for assessing and guiding students’ development. For dental students, the findings of formative assessments are used to diagnose strengths and weaknesses for the purposes of identifying strategies to enhance student performance. For programs, the findings of formative assessments can suggest opportunities to improve the focus of the curriculum and instructional methods.
Measurement:
Measurement refers to the representation of performance using the outcomes of the application of mathematical formulas to numerical data. For many years in the past, classical measurement theory was the primary system used to describe these characteristics or properties. During the last several decades, item response theory (IRT) has been the system of choice for large-scale testing. Because IRT requires large numbers of individuals, however, classical theory remains the mainstay in the academic environment.
Reliability:
Reliability relates to consistency in measurement, i.e., scores derived from a reliable assessment tool are similar across assessment events. Reliability is typically reported as a value ranging from 0.0 to 1.0. Reliabilities above 0.90 are considered to be excellent. Reliabilities below 0.70 are considered suspect, and results from such an assessment tool should be interpreted with caution.
Summative Assessment:
Summative assessment involves the accumulation of information. For the student, the findings of the assessments determine whether the student has accomplished programmatic goals. This form of assessment often represents the level of accomplishment, achievement, or "grade." For the program, findings determine the overall quality of the curriculum for the purposes of making decisions concerning the future of the program.
Validity:
Validity refers to the accumulation of evidence gathered from a variety of sources and supporting the proposition that the assessment is, in fact, evaluating the competency of interest, or the knowledge and abilities that support the acquisition of competence. Evidence can take the form of expert opinion derived from a practice analysis, survey, or standard setting event.
|
Table and Descriptions of Assessment Methods |
|---|
|
TopAuthor informationGlossaryTable and Descriptions of...Written assessments: selected...SourcesAssessment by observationSourcesMultisource assessmentSourcesSimulationSourcesMulti-competency, comprehensive...SourcesWork samplesSourcesGeneral references on assessmentAppendixReferences |
|---|
The separate sections that follow describe the sixteen assessment methods that appear in the summary table. For each method, the characteristics, use, strengths, limitations, and key references are provided. As shown in Figure 1
, the assessment methods are organized into six categories based on the nature of the assessment format and/or the type of response requested from students: 1) selected and constructed response items (written assessment), in which students select a best response from a series of options or construct a response in their own words; 2) faculty assessment by observing student performance, in which students are observed while they perform tasks associated with professional competence by their instructors and are evaluated using checklists and rating scales to guide the appraisal process; 3) multi-source assessment, which refers to a group of assessment tools often used in conjunction with other methods to provide a well-rounded perspective on students’ progression toward competence; 4) simulation, including computer-based applications and realistic models; 5) multi-competency, comprehensive assessments, including the objective structured clinical examination (OSCE) and triple jump exercise (TJE); and 6) work samples, including portfolios and record review.
|
|
Written Assessments: Selected and Constructed Response Items |
|---|
|
TopAuthor informationGlossaryTable and Descriptions of...Written assessments: selected...SourcesAssessment by observationSourcesMultisource assessmentSourcesSimulationSourcesMulti-competency, comprehensive...SourcesWork samplesSourcesGeneral references on assessmentAppendixReferences |
|---|
There are two variations of importance when describing multiple-choice items. The stem is essentially a stimulus for the response. In stand-alone or independent items, the stem is the only stimulus material (Figure 2). In testlet-based or case-based multiple-choice items, additional material is provided as a stimulus (Figure 3). In the case of dental education, the additional stimulus material might consist of a dental chart or history along with a brief clinical scenario, as in a testlet. Case-based items might include radiographs, clinical photographs, and dental charting.
|
|
Strengths and Limitations
This method is widely used because multiple-choice items can be readily developed and scored objectively and can sample widely from an extensive volume of knowledge. This item type has a relatively low level of fidelity to actual practice, however, and it is also susceptible to various measurement disturbances, such as guessing. Additional stimulus material provided in testlet or case-based items enhances the fidelity of the item type and allows for sampling more clinically relevant material from the curriculum.
Short Answer Questions
Description and Characteristics
Stimulus material is provided in a short statement or question that poses a problem requiring the dental students to respond with the solution to the problem in their own words (Figure 4).
|
Strengths and Limitations
By requiring the student to generate a response in the form of a sentence or two, this item type taps different cognitive processes than the multiple-choice item, i.e., it can be used to tap the dental student’s ability to generate an original response. This item type eliminates guessing as an assessment factor; however, evaluation of the response is less objective and is subject to measurement disturbances such as the influence on the rater of spelling, handwriting, and grammatical errors. To help avoid these influences, a key is developed that focuses on the salient information to be provided in the response.
Structured Essay
Description and Characteristics
This item type is similar to the short answer item except that the response presented by the dental student is far more involved. Stimulus material is provided that poses a question or problem. The stimulus material might involve a clinical scenario. The material requires the dental student to provide a logical and detailed response to the question or problem posed (Figure 5).
|
Strengths and Limitations
One of the strengths of this type of item is that it can address most competencies. In the case of competencies associated with patient care, the stimulus material provided involves some sort of clinical scenario. The evaluation of the response can be subjective, however, unless the rater is calibrated, i.e., raters have developed a clear and precise key, or set of criteria, to apply to the response.
|
SOURCES |
|---|
|
TopAuthor informationGlossaryTable and Descriptions of...Written assessments: selected...SourcesAssessment by observationSourcesMultisource assessmentSourcesSimulationSourcesMulti-competency, comprehensive...SourcesWork samplesSourcesGeneral references on assessmentAppendixReferences |
|---|
Downing ST, Haladyna TM, eds. Handbook of test development. Mahwah, NJ: Lawrence Erlbaum, 2006.
Haladyna TM. Developing and validating multiple-choice test items. 3rd ed. Mahwah, NJ: Lawrence Erlbaum, 2004.
Haladyna TM, Downing SM. A taxonomy of multiple-choice item-writing rules. Appl Meas Educ 1989;1:37–50.
|
Assessment by Observation |
|---|
|
TopAuthor informationGlossaryTable and Descriptions of...Written assessments: selected...SourcesAssessment by observationSourcesMultisource assessmentSourcesSimulationSourcesMulti-competency, comprehensive...SourcesWork samplesSourcesGeneral references on assessmentAppendixReferences |
|---|
). This form of assessment requires a clear set of performance indicators or rubrics that are judged to be important relative to the competency or competencies being evaluated. The student is rated on each indicator. The student is typically rated on a scale ranging from 1 to 5, with a 1 indicating that the competency of interest has not been successfully achieved and a 5 indicating most or all aspects of the competency have been achieved successfully. Global ratings are not restricted to a five-point scale, although more than five points tend to be confusing to the rater. For measurement purposes, it is assumed that the scale is comprised of uniformly increasing levels of success.
|
Strengths and Limitations
The strength of this form of assessment lies in its suitability at evaluating general behaviors in a variety of settings. The indicators of the successful acquisition of the competency must be agreed upon by raters as important or critical to the competency. The limitation of the form is the potential for subjectivity in ratings, unless only one rater is used or multiple raters are calibrated to ensure consistency.
Structured Observation with Checklists or Rating Scales
Description and Characteristics
In this type of assessment, students are exposed to a highly structured situation, most often in a clinical setting, with pre-established performance demands. Typically, students are expected to perform a specific skill, and the rating scale includes important components/attributes of that skill rather than a general set of performance parameters as are included on the previous global rating scale. The student’s performance is observed by a rater or multiple raters, who use a checklist to indicate that some aspect of performance has been demonstrated or a set of rating scales to indicate the level of performance relative to the competency or competencies of interest. In the case of a checklist, the agreed upon entries address the critical aspects of the performance. As with global ratings, the student typically is rated on a scale consisting of three, four, or five points, ranging, for example, from 1 to 3, with a 1 indicating that the competency or some component of the competency has not been successfully achieved and a 3 indicating that most or all aspects of the competency have been achieved successfully (Figure 7).
|
Strengths and Limitations
This assessment form is ideal for evaluating specific areas of performance, and it represents a method for evaluating competencies related to the more clinically relevant competencies. As with global ratings, however, it is susceptible to subjectivity on the part of raters.
Standardized Oral Examination
Description and Characteristics
A set of stimulus questions are developed that address critical areas of knowledge or sets of abilities related to a competency or set of competencies. All students being evaluated are exposed to the same set of questions. Students are expected to respond verbally in their own words, which allows an assessment of the student’s depth of comprehension and capacity to apply knowledge and insights to different situations. Responses to the questions are assessed using a standardized rating scale or scoring system.
Uses
This form of assessment is well suited to the evaluation of critical thinking competencies and competencies in the area of professionalism, although it can be used with other competencies exclusive of clinical demonstrations.
Strengths and Limitations
The strength of this form of assessment lies in the opportunity on the part of the rater to examine the underlying logic of the student’s response, assuming that probing for further information is allowed by the design of the examination. As with other faculty assessment methods, however, structured oral examinations are susceptible to subjectivity on the part of raters.
|
SOURCES |
|---|
|
TopAuthor informationGlossaryTable and Descriptions of...Written assessments: selected...SourcesAssessment by observationSourcesMultisource assessmentSourcesSimulationSourcesMulti-competency, comprehensive...SourcesWork samplesSourcesGeneral references on assessmentAppendixReferences |
|---|
Licari FW, Knight GW, Guenzel PJ. Designing evaluation forms to facilitate student learning. J Dent Educ 2008;72(1):48–58.
Ryding HA, Murphy HJ. Employing oral examinations (viva voce) in assessing dental students’ clinical reasoning skills. J Dent Educ 1999;63(9):682–7.[Abstract]
Taleghani M, Solomon ES, Wathen WF. Non-graded clinical evaluation of dental students in a competency-based education program. J Dent Educ 2004;68(6):644–55.
Wagner J, Arteaga S, D’Ambrosio J, Hodge CE, Ioannidou E, Pfeiffer CA, et al. A patient-instructor program to promote dental students’ communication skills with diverse patients. J Dent Educ 2007;71(12):1554–60.
Winckel CP, Reznick RK, Cohen R, Taylor B. Reliability and construct validity of a structured technical skills assessment form. Am J Surg 1994;167:423–7.[Medline]
|
Multisource Assessment |
|---|
|
TopAuthor informationGlossaryTable and Descriptions of...Written assessments: selected...SourcesAssessment by observationSourcesMultisource assessmentSourcesSimulationSourcesMulti-competency, comprehensive...SourcesWork samplesSourcesGeneral references on assessmentAppendixReferences |
|---|
Student Self-Assessment
Description and Characteristics
Dental students evaluate their own performance against a set of criteria related to important competencies or the knowledge, abilities, and skills underlying a competency. This approach might involve a form with rating scales associated with specific criteria. Students might also provide narratives evaluating their performance and reflecting on lessons learned and strategies for enhancement or modification of performance in the future.
Uses
This form of assessment can be applied to any competency or underlying knowledge or ability; however, it is best reserved for assessment of competencies that do not involve demonstrations of highly precise technical skills that are best evaluated by experienced clinicians. Student self-assessment can also be an important component of formative assessment. Student self-assessment has been found to be very useful in the preclinical environment.
Strengths and Limitations
Dental students are best able to assess their own performance if provided with a clearly defined set of criteria against which to make judgments and a set of standards of acceptable levels of knowledge and performance. This form is limited by students’ own ability to view their knowledge or performance realistically. This approach is most effective when used in combination with or comparison to assessment by competent, experienced instructors/evaluators.
Peer Assessment
Description and Characteristics
Dental students assess each other’s performance and, in some instances, knowledge. Checklists or rating scales are best incorporated into this form of assessment.
Uses
This form of assessment is well suited to evaluating the communication and interpersonal skills and health promotion competencies, although peer assessment can be useful with other competencies as well. This assumes the criteria have been clearly articulated and are relevant to essential knowledge and abilities. Peer assessment is also useful in problem-based learning and in other curricular activities in which students’ contributions to group learning are important.
Strengths and Limitations
This approach to the assessment of students is limited, however, by the level of knowledge and ability of the peer group.
Patient Survey
Description and Characteristics
Patient surveys typically involve a series of rating scales or checklists designed to assess the patient’s satisfaction with the performance of the student in the clinical setting. The standard survey solicits the patient’s satisfaction using categories such as poor, good, and excellent, with categories in between where appropriate. Also, categories might be agree, neutral, or disagree with value judgments included on the survey.
Uses
This form of assessment is ideally suited to evaluating those competencies related to communication and interpersonal skills, as well as patient care. However, patient evaluations are not confined to assessing students’ behavior. Perspectives and ratings provided by patients can be expanded to assess the quality of clinic services.
Strengths and Limitations
This form of assessment involves gathering information from patients on those aspects of care that are important to them. It should be possible to complete the assessment in a reasonable amount of time, typically ten to fifteen minutes. This involves interviewing patients and focus groups to determine the critical aspects of care that will be assessed. These aspects, in turn, are translated into a survey form. The survey can be an effective method of gathering information because it is based on the concerns of patients. Limitations include issues related to the patient’s ability to understand the language on the survey, costs involved in obtaining survey information, and the challenges of obtaining a sufficient number of completed surveys to achieve reliable findings.
Standardized Patients
Description and Characteristics
Standardized patients (SPs) are trained individuals who present in a clinical situation with standardized symptoms similar to those that might be encountered with actual patients. These patients present with a full variety of symptoms that allow the dental student to develop a range of treatment plans.
Uses
Standardized patients provide essential feedback on dental student performance, often using a checklist or series of rating scales. Faculty raters also provide feedback in the form of evaluative comments. This is an effective approach to evaluating competencies related to communication and interpersonal skills and patient care.
Strengths and Limitations
This form of assessment provides the students with valuable information on their ability to think critically, their interpersonal skills in working with patients, and their ability to diagnose and develop a treatment plan. It can be subjective, however, unless patients are highly trained and calibrated.
|
SOURCES |
|---|
|
TopAuthor informationGlossaryTable and Descriptions of...Written assessments: selected...SourcesAssessment by observationSourcesMultisource assessmentSourcesSimulationSourcesMulti-competency, comprehensive...SourcesWork samplesSourcesGeneral references on assessmentAppendixReferences |
|---|
Barrows HS. An overview of the uses of standardized patients for teaching and evaluating clinical skills. Acad Med 1993;68:443–51.[Medline]
Butters JM, Willis DO. A comparison of patient satisfaction among current and former dental school patients. J Dent Educ 2000;64(6):409–15.[Abstract]
Curtis DA, Lind SL, Dellinges M, Setia G, Finzen FC. Dental students’ self-assessment of preclinical examinations. J Dent Educ 2008;72(3):265–77.
Ferrell BG. Clinical performance assessment using standardized patients: a primer. Fam Med 1995;27:14–9.[Medline]
Johnson JA, Kopp KC, Williams RG. Standardized patients for the assessment of dental students’ clinical skills. J Dent Educ 1990;54(6):331–3.[Medline]
Thammasitboon K, Sukotjo C, Howell H, Karimbux N. Problem-based learning at the Harvard School of Dental Medicine: self-assessment of performance in postdoctoral training. J Dent Educ 2007;71(8):1080–9.
|
Simulation |
|---|
|
TopAuthor informationGlossaryTable and Descriptions of...Written assessments: selected...SourcesAssessment by observationSourcesMultisource assessmentSourcesSimulationSourcesMulti-competency, comprehensive...SourcesWork samplesSourcesGeneral references on assessmentAppendixReferences |
|---|
Uses
Computer-based scenarios are well suited to assessing competencies associated with diagnosis and treatment planning. These scenarios tend to have high fidelity and make excellent teaching and assessment tools.
Strengths and Limitations
The strength of virtual scenarios lies in their high fidelity. Determining performance levels is difficult, however, and requires considerable research in determining salient decision points in evaluating appropriate diagnoses and treatment plans. This research involves exploring novice and expert performance with focus groups of students and expert practitioners to determine levels of acceptable performance.
Computer-based scenarios that depict actual patient care situations are time-consuming and often expensive to produce, factors reflecting common logistical limitations.
Models
Description and Characteristics
Models consist of mannequins showing various dentally related clinical challenges for the dental student to evaluate.
Uses
This form of assessment taps knowledge and problem-solving skills underlying competencies often related to diagnosis and treatment planning. Because models are standard for all students and evaluation criteria can be readily defined, evaluation can be relatively objective.
Strengths and Limitations
This form of assessment is effective largely because symptoms are easily standardized and consistent across students. Assessing performance is relatively straightforward. It is only limited if criteria are not well defined or those serving as raters are not well calibrated.
|
SOURCES |
|---|
|
TopAuthor informationGlossaryTable and Descriptions of...Written assessments: selected...SourcesAssessment by observationSourcesMultisource assessmentSourcesSimulationSourcesMulti-competency, comprehensive...SourcesWork samplesSourcesGeneral references on assessmentAppendixReferences |
|---|
Jasinevicius TR, Landers M, Nelson S, Urbankova A. An evaluation of two dental simulation systems: virtual reality versus contemporary non-computer-assisted. J Dent Educ 2004;68(11):1151–62.
Littlefield JH, Demps EL, Keiser K, Chatterjee L, Yuan CH, Hargreaves KM. A multimedia patient simulation for teaching and assessing endodontic diagnosis. J Dent Educ 2003;67(6):669–77.[Abstract]
Wierinck ER, Puttemans V, Swinnen SP, van Steenberghe D. Expert performance on a virtual reality simulation system. J Dent Educ 2007;71(6):759–66.
|
Multi-Competency, Comprehensive Assessments |
|---|
|
TopAuthor informationGlossaryTable and Descriptions of...Written assessments: selected...SourcesAssessment by observationSourcesMultisource assessmentSourcesSimulationSourcesMulti-competency, comprehensive...SourcesWork samplesSourcesGeneral references on assessmentAppendixReferences |
|---|
Uses
This form of assessment provides a standardized opportunity for students to demonstrate their ability to conduct an oral examination, take comprehensive notes, and interpret the clinical situation. This is an excellent format for evaluating a full range of competencies, especially those related to diagnosis and treatment.
Strengths and Limitations
The strengths of this form of assessment are that it provides an opportunity for students to demonstrate specific clinical skills and it has high fidelity. It is also effective at evaluating competencies related to treatment. However, gathering reliable performance information can be problematic since there are a limited number of stations. It is logistically difficult and time-consuming to develop and conduct this form of assessment.
Triple Jump Exercise (TJE)
Description and Characteristics
There are two types of TJEs used in different components of the curriculum, but they involve similar techniques. A clinical TJE consists of three phases ("jumps") in which students 1) interview and examine patients while observed by faculty or, less often, are videotaped for retrospective review; 2) write an assessment of the findings from the patient assessment using the "SOAP" format (subjective data, objective data, assessment, plans), with emphasis on providing evidence from the literature to support diagnostic and therapeutic decisions and submit this document to the faculty member who observed jump one; and 3) participate in an oral examination conducted by the observing faculty member in which students are questioned about the pathophysiology, diagnosis, and treatment of the patient’s problems and asked to discuss research evidence pertinent to treatment and outcomes. Students receive an evaluation for each jump and a cumulative score across all three jumps.
TJEs implemented in the preclinical curriculum focus on students’ skills in searching the literature to answer health-related questions that they have developed. In a preclinical TJE, the first jump involves reading a scenario depicting a patient with an oral health problem, identifying key issues, and writing a researchable question in the PICO format (patient with problem, intervention, comparison, and outcome). During the second jump, students investigate literature to find evidence pertinent to their question, and then, in jump three, report their findings, answer the research question, and critically appraise the quality of available evidence. As with clinical TJEs, preclinical students receive evaluations for each jump and a cumulative score for the whole exercise.
Uses
The triple jump exercise is used to evaluate students’ capacity to access, analyze, and apply biomedical knowledge to health care problems. When coupled with multiple-choice testing in the case-based testlet format (i.e., several multiple-choice questions linked to a patient scenario), TJEs provide a mechanism for assessment of students’ capacity to function at the application level of the cognitive taxonomy. Both types of TJEs emphasize accessing pertinent information, applying this information to health problems, and appraising the quality of knowledge available to answer clinical questions.
Strengths and Limitations
The strength of this form of assessment is that it provides an opportunity for faculty to appraise the student’s performance across a spectrum of skills ranging from patient assessment (conducting an interview or performing an examination) to diagnosis and treatment planning and ultimately the ability to explain the rationale for selected therapy and demonstrate understanding of the research evidence pertinent to the patient’s oral health problems and therapeutic options. However, the TJE for either preclinical or clinical students is time-consuming to develop and logistically difficult to implement, requiring considerable numbers of faculty members who need to be trained in the technique and calibrated to provide uniform assessments of each of the jumps.
|
SOURCES |
|---|
|
TopAuthor informationGlossaryTable and Descriptions of...Written assessments: selected...SourcesAssessment by observationSourcesMultisource assessmentSourcesSimulationSourcesMulti-competency, comprehensive...SourcesWork samplesSourcesGeneral references on assessmentAppendixReferences |
|---|
Gerrow JD, Murphy HJ, Boyd MA, Scott DA. Concurrent validity of written and OSCE components of the Canadian dental certification examinations. J Dent Educ 2003;67(8):896–901.[Abstract]
Rangachari PK. The TRIPSE: a process-oriented evaluation for problem-based learning courses in the basic sciences. Biochem Mol Biol Educ 2002;30(1):57–60.
Schoonhein-Klein M. The use of the objective structured clinical examination (OSCE) in dental education. Amsterdam: M.E. Schiinheim-Kleub, 2007.
Smith RM. The triple-jump examination as an assessment tool in the problem-based medical curriculum at the University of Hawaii. Acad Med 1993;13:366–72.
|
Work Samples |
|---|
|
TopAuthor informationGlossaryTable and Descriptions of...Written assessments: selected...SourcesAssessment by observationSourcesMultisource assessmentSourcesSimulationSourcesMulti-competency, comprehensive...SourcesWork samplesSourcesGeneral references on assessmentAppendixReferences |
|---|
Uses
This is a student-centered tool, which is an effective method for evaluating a variety of competencies such as critical thinking, professionalism, and health promotion.
Strengths and Limitations
The strength of this assessment tool is that it allows for the evaluation of competencies that are not readily evaluated using other tools. However, the variability among portfolios makes consistent evaluation difficult for raters. Also, supporting students in their development of their portfolios can be time-consuming, as can evaluating the documents themselves. It has been found that, because of these limitations, portfolios can be problematic to both students and faculty raters.
Record Review (Chart-Stimulated Review)
Description and Characteristics
This methodology involves a review of the patient care records (i.e., patients’ charts) developed by the student. The review consists of an evaluation of diagnostic information and an examination of findings related to treatment planning in light of standards of dental practice. Chart-stimulated review is commonly used as an assessment technique in medical education for residents and students on clerkships. The method assesses the learner’s capacity to explain rationales for treatment decisions, show comprehension of key concepts, and compare and contrast alternative treatment approaches; it also is used to stimulate students’ self-assessment and reflection.
Uses
Record review is effective for evaluating competencies that are not readily assessed by other tools including competencies in the domains of critical thinking, professionalism, and health promotion in the ADEA Competencies for the New General Dentist.
Strengths and Limitations
Similar to portfolios, a wide range of competencies can be evaluated with chart reviews. The variability among records can lead to subjective judgments regarding their quality, however.
|
SOURCES |
|---|
|
TopAuthor informationGlossaryTable and Descriptions of...Written assessments: selected...SourcesAssessment by observationSourcesMultisource assessmentSourcesSimulationSourcesMulti-competency, comprehensive...SourcesWork samplesSourcesGeneral references on assessmentAppendixReferences |
|---|
Friedman Ben David M, Davis M, Harden R, Howie P, Ker J, Pippard M. Portfolios as a method of student assessment. AMEE Medical Education Guide No. 24. Med Teacher 2001;23(6):535–51.
Logan H, Gardner T. A review of a dental record audit program within a predoctoral dental curriculum. J Dent Educ 1988;52(6):302–5.[Medline]
|
General References on Assessment |
|---|
|
TopAuthor informationGlossaryTable and Descriptions of...Written assessments: selected...SourcesAssessment by observationSourcesMultisource assessmentSourcesSimulationSourcesMulti-competency, comprehensive...SourcesWork samplesSourcesGeneral references on assessmentAppendixReferences |
|---|
American Educational Research Association, American Psychological Association, National Council on Measurement in Education. Standards for educational and psychological testing. Washington, DC: American Educational Research Association, American Psychological Association, National Council on Measurement in Education, 1999.
The standards are currently under review, and a revised set of standards will be published in the near future.
Guides to Assessment Techniques in Health Professions Education
Joint Commission on National Dental Examinations. Case development guide: National Board Dental Examination. Chicago: American Dental Association. 2007. At: www.ada.org/prof/ed/testing/construction/nbde02_case_guide.pdf.
Joint Commission on National Dental Examinations. Case development guide: National Board Dental Hygiene Examination. Chicago: American Dental Association, 2007. At: www.ada.org/prof/ed/testing/construction/nbdhe_case_guide.pdf.
Joint Commission on National Dental Examinations. Test item development guide: National Board Dental Examination. Chicago: American Dental Association, 2007.
National Board of Medical Examiners. Subject examinations: content outlines and sample items. Philadelphia: National Board of Medical Examiners, 2003:1–38.
National Board of Medical Examiners. Section II: writing one-best-answer questions for the basic and clinical sciences. Philadelphia: National Board of Medical Examiners, 2003:33–67.
National Board of Medical Examiners. Section III: extended matching items. Philadelphia: National Board of Medical Examiners, 2003:71–103.
National Board of Medical Examiners. Appendix B. Sample item-writing templates, items, lead-ins, and option lists for the basic and clinical sciences. Philadelphia: National Board of Medical Examiners, 2003:131–80.
Textbooks and Monographs
Downing ST, Haladyna TM, eds. Handbook of test development. Mahwah, NJ: Lawrence Erlbaum, 2006.
Fortune JC, Cromack TR. Developing and using clinical examinations. In: Impara JC, ed. Licensure testing: purposes, procedures, and practices. Lincoln, NE: Buros Institute of Mental Measurements, 1995:149–65.
Haladyna TM. Developing and validating multiple-choice test items. 3rd ed. Mahwah, NJ: Lawrence Erlbaum, 2004.
Irvine SH, Kyllonen PC, eds. Item generation for test development. Mahwah, NJ: Lawrence Erlbaum, 2002.
LaDuca A, Downing SM, Henzel TR. Systematic item writing and test construction. In: Impara JC, ed. Licensure testing: purposes, procedures, and practices. Lincoln, NE: Buros Institute of Mental Measurements, 1995:117–48.
Middle States Commission on Higher Education. Student learning assessment: options and resources. Philadelphia: Middle States Commission on Higher Education, 2003.
Schmeiser CB, Welch CJ. Test development. In: Brennan RL, ed. Educational measurement. 4th ed. Washington, DC: American Council on Education and Praeger Publishers, 2006:307–54.
Journal Articles: Summaries of Assessment Best Practices
Azer SA. Assessment in a problem-based learning course. Biochem Mol Biol Educ 2003;31(6):248–434.
Chambers DW, Glassman P. A primer on competency-based evaluation. J Dent Educ 1997;61(8):651–66.
Epstein RM. Assessment in medical education. N Engl J Med 2007;356(4):387–96.
Epstein RM, Hundert EM. Defining and assessing professional competence. JAMA 2002;287(2):226–35.
Friedman Ben David M, Davis M, Harden R, Howie P, Ker J, Pippard M. Portfolios as a method of student assessment. AMEE medical education guide no. 24. Med Teacher 2001;23(6):535–51.
Hoge R, Coladarci T. Teacher-based judgments of academic achievement: a review of the literature. Rev Educ Res 1989;59(3):297–313.
Linn RL. Complex, performance-based assessment: expectations and validation criteria. Educ Researcher 1991;16(1):1–21.
Miller GE. Assessment of clinical skills/competence/performance. Acad Med 1990;9:63–7.
Norcini J, Burch V. Workplace-based assessment as an educational tool. AMEE guide no. 31. Med Teacher 2007;29(9 & 10):855–71.
Shumway JM, Harden RM. The assessment of learning outcomes for the competent and reflective physician. AMEE guide no. 25. Med Teacher 2003;25(6):569–84.
Smith SR, Dollase RH, Boss JA. Assessing students’ performance in a competency-based curriculum. Acad Med 2003;78:97–107.
Swing SR. Assessing the ACGME general competencies: general considerations and assessment methods. Acad Emerg Med 2002;9(11):1278–87.
Van der Vieuten CPM. The assessment of professional competence: developments, research, and practical implications. Adv Health Sci Educ 1996;1:41–67.
Van der Vieuten CPM, Schuwirth L. Assessing professional competence: from methods to programmes. Med Educ 2005;39:309–17.
Wass V, Van der Vieuten CPM, Shatzer J, Jones R. Assessment of clinical competence. Lancet 2001;357(March 24):945–9.
Other Resources
Paul R, Nosich GM. A model for the national assessment of higher order thinking, section four. Critical thinking community, 1993. At: www.criticalthinking.org/assessment/a-model-nalassessment-hot.cfm.
The assessment toolkit developed by the Accreditation Council for Graduate Medical Education can be accessed at www.acgme.org/Outcome/assess/Toolbox.pdf.
Organizations Involving Assessment
American Dental Education Association, Washington, DC. At: www.adea.org.
American Educational Research Association, Washington, DC. At: www.aera.net.
American Psychological Association, Washington, DC. At: www.apa.org.
Association for Psychological Science, Washington, DC. At: www.psychologicalscience.org.
National Council on Measurement in Education, Washington, DC. At: www.ncme.org.
|
APPENDIX |
|---|
|
TopAuthor informationGlossaryTable and Descriptions of...Written assessments: selected...SourcesAssessment by observationSourcesMultisource assessmentSourcesSimulationSourcesMulti-competency, comprehensive...SourcesWork samplesSourcesGeneral references on assessmentAppendixReferences |
|---|
Toolbox of assessment techniques to measure dental students’ attainment of the 2008 ADEA Competencies for the New General Dentist (1: Preferred; 2: Acceptable; 3: Potentially Applicable)
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Author Information |
|---|
|
TopAuthor informationGlossaryTable and Descriptions of...Written assessments: selected...SourcesAssessment by observationSourcesMultisource assessmentSourcesSimulationSourcesMulti-competency, comprehensive...SourcesWork samplesSourcesGeneral references on assessmentAppendixReferences |
|---|
This article is one in a series of invited contributions by members of the dental education community that have been commissioned by the American Dental Education Association’s Commission on Change and Innovation in Dental Education (ADEA CCI) to address the environment surrounding dental education and affecting the need for, or process of, curricular change. This article was written at the request of the ADEA CCI but does not necessarily reflect the views of ADEA, the ADEA CCI, or individual members of the ADEA CCI. The perspectives communicated here are those of the authors.
|
REFERENCES |
|---|
|
TopAuthor informationGlossaryTable and Descriptions of...Written assessments: selected...SourcesAssessment by observationSourcesMultisource assessmentSourcesSimulationSourcesMulti-competency, comprehensive...SourcesWork samplesSourcesGeneral references on assessmentAppendixReferences |
|---|
This article has been cited by other articles:
|
|
 |
|
  J. Phillips and Z. L. Berge Second Life for Dental Education J Dent Educ., November 1, 2009; 73(11): 1260 - 1264. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
