JDE
HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS
QUICK SEARCH:   [advanced]


     

J Dent Educ. 70(5): 500-510 2006
© 2006 American Dental Education Association
This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Taleghani, M.
Right arrow Articles by Wathen, W. F.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Taleghani, M.
Right arrow Articles by Wathen, W. F.

Educational Methodologies

Grading Dental Students in a "Nongraded" Clinical Assessment Program

Mohsen Taleghani, D.M.D.; Eric S. Solomon, D.D.S., M.A.; William F. Wathen, D.M.D.

Key words: clinical evaluation, nongraded assessment, grade point average, competency-based dental education, quality assessment, quantity assessment

Submitted for publication 05/17/05; accepted 01/03/06


   Abstract
Top
 Abstract
Program description
 Discussion
 References
 
This article details how Baylor College of Dentistry (BCD) merges graded and nongraded aspects of student assessments into grade point averages (GPAs); explains the use of its assessment tools to evaluate students, faculty, and curriculum simultaneously; and calls for continuous progress and quality improvement toward educational excellence among all levels of the oral health education community. "Nongraded" student assessments at BCD are only applicable to preclinical and chairside activities that involve direct patient care. We further summarize how we have attempted to improve the accuracy of grading while providing objective numeric grades to measure student performance and generate a class rank. We suggest that faculty evaluation and curricular change require appropriate evaluation methods and continuous quality improvement based on the chosen assessment methodologies. We then summarize how we merge graded and nongraded assessments into a final student evaluation that realistically discriminates among students’ performance and present our numeric-to-letter grade conversion table. We conclude that 1) multiple strategies are required, available, and adequate to provide graduates with the numerical GPAs demanded by postdoctoral programs and that 2) continuous quality improvement among all levels of the oral health education community should be vigorously pursued by administration and faculty alike.

Multiple calls for evolutionary change in oral health education strategies have been made in the past decade.1–3 There are major and growing efforts throughout the dental educational enterprise to address these broad concerns. The move from discipline-based to competency-based systems in dental education has required rethinking of the entire educational spectrum.4–6 New systems of performance assessment for students, faculty, and administrators alike are necessary.7,8 New educational methodology is replacing the teacher-driven pedagogic styles of discipline-based systems.9 A new U.S. dental school has based its predoctoral program upon innovative educational methodology,10 reflective of the transition toward providing students a sequential series of experiences aimed at clinical competencies. Many schools have incorporated simulation laboratories and multimedia lecture facilities that enhance the "hear it, see it, do it" learning sequence. Fundamentals of adult learning appropriate to self-directed professional development are becoming better understood and implemented among dental educators.11–14 Institutional assessment and continuous quality improvement strategies are woven into all of these references as our schools search for improved risk management, health care delivery, and higher accreditation standards.

Parts of traditional strategies remain useful. Among the attributes of discipline-based, teacher-controlled instructional objectives is the numeric grade, which serves as a means of assessing and relatively ranking students. Alpha-numeric systems accomplish this task quite well and are appropriately utilized in the traditional didactic courses for novice and beginning learners. Based upon outcome measures of our system at Baylor College of Dentistry (BCD), it is in clinical decision making, professional behavior, time/patient management, and actual delivery of clinical procedures that nongraded assessment seems to offer a better solution for student and teacher alike.7

Students seeking admission to advanced educational programs must have high ranking based upon objective measures if they are to be competitive. The entire educational enterprise has a continuing responsibility to ensure that those rankings are both valid and reliable and that they indeed discriminate among learners, both clinically and nonclinically, as to total performance over time. Therefore, it seems that combining graded and nongraded strategies as we assess and differentiate our students offers a more collegial and balanced system.

This report follows a previous publication7 about our nongraded clinical assessment strategy for evaluating professional behavior, clinical competencies, and graduation requirements for our students. In this article, we detail how we arrive at a traditional grade point average in our nongraded system.


   Program Description
Top
 Abstract
 Program description
Discussion
 References
 
Student performance in the senior year at BCD is evaluated both subjectively and objectively in all clinical areas by:

  1. nongraded clinical evaluation of chairside performance,7
  2. comprehensive care program group leader conferences,
  3. semester progress summary reports,
  4. a "relative value point" system,
  5. progress examinations,
  6. competency examinations,
  7. professional performance,
  8. program requirements, and
  9. final grade tabulation.

Nongraded Clinical Evaluation of Chairside Performance
As previously reported,7 assessment forms have been developed that allow critical procedural steps to be evaluated in terms of probable clinical success. For example, Figure 1Go displays the form used for direct/indirect restoration quality assessment. Schoolwide, there are thirteen different forms, specific for each clinical discipline. The general dentistry department oversees all senior student clinical activities, but primarily administers only two of the forms: oral diagnosis and treatment planning, and direct/indirect restoration. Other departments administer the remainder. A detailed General Dentistry Faculty Calibration Manual has been developed to enhance consistency among faculty members, who evaluate each procedural step to assess for quality control and clinical acceptability. Every faculty member contributes to and is responsible for knowing the contents of the manual, which is regularly discussed and updated as needed in weekly departmental meetings. Quality assessment (QA) comments are entered on the forms under the "comments" section for daily input into the central database of our clinical management system (Axium). (See Figure 2Go.) This in turn provides on-demand feedback and tracking of both student and faculty performance, which is used for our continuous quality improvement and curriculum evaluation programs.


Figure 1
View larger version (60K):
[in this window]
[in a new window]
  		Figure 1. Direct/indirect restoration quality assessment sheet

 

Figure 2
View larger version (41K):
[in this window]
[in a new window]
  		Figure 2. Sample QA assessment comment summary printout

 
Student performance deficiencies are thereby noted and summarized daily, enabling immediate specific remediation by group leaders or other appropriate faculty if required. An example of such student remediation occurs in repeated instances of unidentified residual caries in a preparation. Direct supervision of identification and removal of caries takes place chairside in the clinic and is followed up in remediation sessions by directly supervised caries identification and removal on extracted teeth. Monthly printout summaries list all evaluations done by each faculty member, thus providing a mechanism to compare individual faculty performances against the group range and averages.

Comprehensive Care Program (CCP) Group Leader Conferences
The senior class at BCD is divided into groups of approximately fourteen students and assigned to a general dentistry faculty member who serves as their group leader for the three terms of the senior year. That person becomes the student’s mentor, advocate, cheerleader, disciplinarian, and remedial resource director.

Remediation may involve other individuals or departments, is specific to the identified problem, and is supervised by the student’s group leader in consultation with the department chair. Regular meetings are held monthly (and as otherwise needed) between group leaders and each student to review performance and other issues that arise.

Each group has a patient appointment associate (PAA) assigned to assist the students and group leader with administrative matters. This individual serves as the office manager would in private practice, while the business office of the college manages financial arrangements. Daily reports from PAAs and the business office are entered into each student’s database, and weekly printouts of every student’s activities and evaluations are given to group leaders. These results summarize every quality assessment (QA) mark entered on daily assessment sheets, and the results are discussed in group leader/student conferences. QA marks are indications of substandard performance of a particular step. Thus, daily QA check summaries can be combined into individual student trends so that areas of weakness can be objectively identified and remediated. The results are then tabulated on student conference forms (Figure 3Go).


Figure 3
View larger version (19K):
[in this window]
[in a new window]
  		Figure 3. Student conference form

 
Progress Reports
Progress reports are end-of-semester compilations of daily, weekly, and monthly reports. Daily assessment sheets are two-part forms, one of which is given to the student at the conclusion of a procedure. Group leaders receive weekly summaries of the assessment sheets and arrange conferences as needed with each student. At least once a month a general conference is completed and recorded (Figure 3Go). Student conference results are then summarized in progress reports.

Students receive a copy of their progress reports at the end of the summer, fall, and spring (final) semesters (see Figure 4Go). Satisfactory progress is required in all areas. Students failing to meet any of the criteria are remediated by their group leader or other appropriate designee. Additional patients and/or laboratory exercises may be assigned, along with any other remedial activities appropriate to the case. Correction of unsatisfactory professional behavior can range from case-specific remediation activities to temporary suspension of clinical privileges and, in extreme cases, to dismissal from BCD.


Figure 4
View larger version (17K):
[in this window]
[in a new window]
  		Figure 4. General dentistry progress summary form

 
Relative Value Point System
Fourth-year students are accountable for all CCP clinic hours. Just as office overhead is a constant of private practice, students are required to maintain a minimum "overhead factor" of twelve points per available hour of clinic time. Every procedure or activity related to BCD is assigned a relative value that students "earn" by participation (see Figure 5Go). In this manner, quantity of work is assessed along with quality as already discussed. Relative value point (RVP) deductions may be incurred for errors, time management faults, and other minor infractions. One-half the fourth-year grade is based on the student’s RVP (see Figure 6Go).


Figure 5
View larger version (59K):
[in this window]
[in a new window]
  		Figure 5. Sample relative value points for selected clinical procedures

 

Figure 6
View larger version (9K):
[in this window]
[in a new window]
  		Figure 6. Relative value point (RVP)/grade conversion table

 
Progress Examinations
These periodic laboratory and clinical examinations comprise 30 percent of the final grade and cover restorative (operative and fixed prosthodontics) dentistry, endodontics, periodontics, oral and maxillofacial surgery, and removable prosthodontics. Fourth-year students must pass all parts of these examinations to qualify for graduation. The department of general dentistry directs restorative progress examinations, and the other clinical progress examinations are administered by the respective departments.

Restorative progress examinations consist of simulation laboratory exercises early in the summer session of the senior year, followed by patient exercises in the fall and spring clinics. The simulation laboratory exercises consist of full cast crown preparations, posterior Class II preparation, fill, and finish (both amalgam and composite resin), and Class III composite preparation, fill, and finish.

The clinical progress examinations begin in the fall semester, using the student’s selected (with group leader approval) patients of record. Procedures consist of a posterior full cast crown, impression, pour, die trim, wax, cast, polish, and seat; Class III composite preparation, fill, and finish; Class II posterior composite preparation, fill, and finish; and Class II posterior amalgam preparation, fill, and finish. Progress examinations in other clinical disciplines of periodontics, removable prosthodontics, and oral surgery are administered by the respective departments.

No faculty assistance or advice is offered during any of these tests, and the results are graded remotely and objectively by faculty of the appropriate department in a blinded fashion after the Western Regional Examining Board (WREB). For details of WREB protocols, see www.wreb.org. Remediation of deficient performance is initiated immediately by repetition of the procedures deemed below clinical acceptability until satisfactory performance levels are achieved.

Competency Examinations
The three-day competency examinations (mock boards) are administered late in the spring semester, account for 20 percent of the final general dentistry grade, and are structured to duplicate the clinical/laboratory portions of the WREB. These examinations include restorative, periodontic, removable prosthodontic, and endodontic exercises. This experience improves the student’s ability to judiciously select patients and identify lesions, and compels them to perform required procedures within the allotted time and without faculty assistance. Performance on this examination is also objective and blinded, duplicates the WREB protocol, and involves all appropriate departments. Feedback is immediate, and deficient performance is remediated immediately as described in the previous section.

Professional Performance
Every student is expected to consistently demonstrate intellectual, ethical, and behavioral attributes of professionalism (see Figure 1Go, lines 1–17). Unsatisfactory progress results in penalties ranging from remediation to temporary suspension of clinic privileges to permanent dismissal from the General Dentistry CCP. Every assessment made of student performance has a professionalism component as shown in Figure 1Go. Quality assessment checks are reported on weekly, monthly, and semester progress reports, and deficiencies are remediated immediately to correct and avoid behavior that places the student at risk for more severe penalties. Repeated professional performance shortcomings are discussed in weekly faculty meetings and referred to the dean for student affairs by the chairman of general dentistry.

Program Requirements
The requirements for certification for graduation are competency confirmation from all departments involved in the CCP program and 1) completed treatment of all assigned patients as verified by group leaders; 2) minimum production of twelve relative value points per hour of available clinic time (see Figures 4Go and 7Go); 3) successful participation in all remedial, progress, and competency examinations; and 4) 90 percent attendance in available clinic hours.


Figure 7
View larger version (13K):
[in this window]
[in a new window]
  		Figure 7. Final grade tabulation form

 
Final Grade Tabulation
Figure 7Go shows the summary form used for tabulation of each senior student’s final numeric and letter grades in general dentistry. It is not coincidental that the first consideration is fulfillment of our behavioral competencies (see lines 1 through 17 on Figure 1Go). Professional behavior is deemed the most important component of performance as doctors. Students who do not consistently demonstrate the ethical and moral imperatives of the profession are not certified for graduation, regardless of meeting other requirements. The second line of Figure 7Go considers the program requirements listed in the preceding paragraph, and again, failure to achieve an affirmative answer on this line results in failure to achieve certification for graduation. If both lines one and two are favorable, numerical scores are calculated and weighted as shown in Figure 7Go, and a final letter grade is calculated.

Curriculum and Faculty Evaluation
Daily data entry and summary reports pinpoint both curriculum weaknesses and faculty performance. For example, recent faculty discussions have been based on numerous QA checks on daily assessment sheets in both diagnostic recognition (Oral Diagnosis and Treatment Planning Quality Assessment sheet, not shown) and caries removal (Figure 1Go, Line 26) lines. This increase in QA checks reflected a general uncertainty among new senior students about caries detection radiographically, preoperatively, and intraoperatively. All concerned faculty have participated in these discussions for corrective teaching, and next year’s data will be monitored closely to verify that remedial curricular and teaching changes have been effective.

Monthly printouts of faculty performance monitor individual tendencies as to both scope and scale of grading student performance. Numbers of and reasons for QA checks are reported, and faculty are then able to compare their individual performance against the group averages. One of the authors, for example (WFW), found it very difficult to abandon the previous "glance and grade" system of noting a deficiency, discussing it with the student, and moving on to the next student without recording the event. After several months of self-administered comparisons, both quantitative and qualitative aspects of his clinical performance versus the group averages have improved. That corrective behavior change has resulted in a more precise detailing and recording of student performance.


   Discussion
Top
 Abstract
 Program description
 Discussion
References
 
Use of the term "nongraded assessment" can be misinterpreted. Objective numeric grades are both necessary and desirable in courses in which performance can be adequately and objectively assessed. There is a compelling need to identify and rank low and high performers objectively. However, when students move into advanced courses and clinical experiences that require performance related to critical thinking and clinical decision making, as well as the execution of clinical procedures, numeric grading becomes increasingly subjective, arbitrary, and therefore problematic. We have previously described how we developed a nongraded assessment system for these instances of critical thinking and professional judgment.7

Periodic objective graded student assessments are administered via criterion-based, blinded assessment of performance on progress examinations, remedial assignments, competency examinations, and a "relative value point" system that rewards productivity and efficient time management. The combination of graded and nongraded exercises results in a final overall grade for the senior year, balanced as to both quality and quantity of the year’s activities. The blinded assessments in progress and competency examinations have resulted in a broader range of grades among students. The tendency towards grading leniency has been eliminated, and faculty discussions with students are more collegial and less stressful for both parties since alpha-numeric grades no longer are used to assess clinical competency on a daily basis.

The Department of General Dentistry is responsible for the senior year experience and, as such, has served as the model for the revised BCD grading system. In bringing the BCD mission of "developing exemplary clinicians" (see bcd.tamhsc.edu/Mission_Goals/mission_goals.html for the complete BCD mission statement) to a successful reality and with consideration of the multiple calls for educational reform, the department promulgates a patient-centered, competency-based program of comprehensive dental care in an environment resembling private general practice. We are able to simulate features like overhead costs, patient and staff management, treatment planning and execution, and time management for effectiveness. We are not able to simulate features like the financial responsibilities and implications of owning a practice; hiring, training, and firing employees; developing and executing a successful business plan and office management scheme; or creating and maintaining a consulting team.

The overall success or failure of our system will be determined over time, but we believe the combination of additional multiple data into a final tabulation of student performance (Figure 7Go) increases both the validity and reliability of our time-honored and accepted traditional assessment strategies.

The intended outcome of our program is to advance the quality and effectiveness of teaching and to consistently produce ethical graduates who are diagnostically, managerially, and therapeutically competent. In the senior year, we expect our students to competently diagnose and provide comprehensive oral health care in a professional manner.

All activities of the senior year are expected to demonstrate in word and actions the profession’s commitment to provide competent care in a timely manner while respecting the patient’s values and interests. Thus our student’s final year reinforces the role of the general dentist as attending doctor, capable provider of services, patient advocate, team leader, exemplar of the profession, and vital community asset.

Our continuous quality assessment efforts allow us to monitor all these desired outcomes and also serve as an integral part of curricular evaluation and change by documenting weaknesses in senior student knowledge levels. The multiple feedback loops created in our integrated assessment system also provide a basis for faculty, staff, and administrative change, both qualitatively and quantitatively. Thus, continuous assessment and quality improvement, central to the educational process, is an enterprise-wide affair. Implementation of contemporary best practices educational strategies inevitably means change and is vital to the inclusive learning culture each of our organizations must build and nurture.14


   Footnotes
 
Dr. Taleghani is Professor and Chair, Department of General Dentistry, Baylor College of Dentistry, The Texas A&M University System Health Science Center; Dr. Solomon is Executive Director for Institutional Research, The Texas A&M University System Health Science Center, and Professor, Department of Public Health Sciences, Baylor College of Dentistry; and Dr. Wathen is Associate Professor, Department of General Dentistry, Baylor College of Dentistry, The Texas A&M University System Health Science Center. Direct correspondence and reprint requests to Dr. Mohsen Taleghani, Department of General Dentistry, Baylor College of Dentistry, 3302 Gaston Avenue, Room 313, Dallas, TX 76246; 214-828-8414 phone; 214-828-8952 fax; mtaleghani{at}bcd.tamhsc.edu.


   REFERENCES
Top
 Abstract
 Program description
 Discussion
 References
 

  1. Dental education at the crossroads. An Institute of Medicine Report. Washington, DC: National Academy Press, 1995:94.
  2. American Dental Association. Future of dentistry. Chicago: American Dental Association, Health Policy Resources Center, 2001:88–113.
  3. DePaola DP, Slavkin HC. Reforming dental health professions education: a white paper. J Dent Educ 2004; 68(11):1139–50.[Abstract/Free Full Text]
  4. Commission on Dental Accreditation. Accreditation standards for dental education programs. Chicago: American Dental Association, 2004.
  5. Chambers DW. Toward a competency-based curriculum. J Dent Educ 1993;57(11):790–3.[Medline]
  6. Chambers DW, Glassman P. A primer on competency-based evaluation. J Dent Educ 1997;61(8):651–66.[Medline]
  7. Taleghani M, Solomon ES, Wathen WF. Nongraded clinical evaluation of dental students in a competency-based education program. J Dent Educ 2004;68(6):644–55.[Abstract/Free Full Text]
  8. Yip H-K, Smales RJ, Newsome PRH, Chu FCS, Chos TW. Competency-based education in a clinical course in conservative dentistry. Br Dent J 2001;191(9):517–22.[Medline]
  9. Berk N. Teaching ethics in dental schools: trends, techniques, and targets. J Dent Educ 2001;65(8):744–50.[Abstract]
  10. Sanders RM, Ferrillo PJ Jr. A new school’s perspective on clinical curriculum. J Dent Educ 2003;67(12):1316–9.[Abstract]
  11. Knowles MS. The modern practice of adult education: androgogy vs pedagogy. Chicago: Follett Publishing Company, 1970.
  12. Candy PC. Self-direction for lifelong learning. San Francisco: Jossey-Bass Publishers, 1992.
  13. Brookfield SD. Understanding and facilitating adult education. San Francisco: Jossey-Bass Publishers, 1992.
  14. Masella RS, Thompson TJ. Dental education and evidence-based educational best practices: bridging the great divide. J Dent Educ 2004;68(12):1266–71.[Abstract/Free Full Text]




This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Taleghani, M.
Right arrow Articles by Wathen, W. F.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Taleghani, M.
Right arrow Articles by Wathen, W. F.

HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS