Adjusting for hard and easy graders

The dental education literature demonstrates that rater training and the use of criterion-anchored evaluation systems have been only partially successful in reducing the use of private standards by raters. A method is described for statistically removing this source of bias. A study is reported where scores normalized using this technique were significantly better predictors of students' future performance than were raw scores. Assumptions, techniques, and practical considerations are also discussed.