J Dent Educ. 73(3): 287-302 2009
© 2009 American Dental Education Association
Critical Issues in Dental Education |
Issues in the Interpretation and Reporting of Surveys in Dental Education
David W. Chambers, Ed.M., M.B.A., Ph.D.;
Frank W. Licari, D.D.S., M.P.H., M.B.A.
Key words: surveys, competency-based education, standard error, precision, accuracy, bias
Submitted for publication 08/01/08;
accepted 12/09/08
Surveys are the most common form of data-based article published in the Journal of Dental Education. The apparent ease with which they are conducted and the seeming simplicity of reporting results mask significant issues in sample design and performing maximally useful analyses. Four concerns are discussed here. First, it is demonstrated that results are a function of who, when, and where responses are sampled, each source making independent contributions. Second, absolute sample size is shown to be the most significant factor affecting precision in surveys, and the numbers of schools, respondents, and other sources of variance can be chosen to minimize survey imprecision. Third, response rate typically has negligible effect on precision and an uncertain effect on accuracy (freedom from bias). A technique, sample saturation, is explained that can be used to protect, to some degree, surveys from the effects of bias. Finally, suggestions are offered for reporting survey results in a visually meaningful fashion, and an appeal is made that recommendations associated with surveys not be published unless they are grounded in both data and well-developed theory. This analysis references a previously published survey on competency-based dental education to illustrate methodological points in concrete terms.
Copyright © 2009 by the American Dental Education Association.