|
|
||||||||
Critical Issues in Dental Education |
Key words: Dental Admission Test, dental school performance, admissions criteria, underachieving students
Submitted for publication 03/29/07; accepted 07/23/07
| Abstract |
|---|
|
|
|---|
Admissions criteria have traditionally included college grade point average (GPA), science grade point average (SGPA), the overall Dental Admission Test (DAT), and components of the DAT including academic average (AA) and the Perceptual Ability Test (PAT).1–3,6,7 Less frequently used criteria have included the academic rigor of the college attended,8 the average academic load while in college,6 psycho-motor assessments such as chalk carving,9 waxing tests,10,11 and various behavioral measures of empathy and skill in verbal communication.11,12
Standardized tests generally predict other standardized tests better than they predict noncognitive outcome measures because similar testing formats generally increase the validity of compared assessments.5,13 This is also true in evaluating admissions criteria commonly used in dental school admissions, which often fail to strongly predict both cognitive and psychomotor achievement.1,2,6 For example, the Perceptual Ability Test (PAT) of the DAT is a better predictor of preclinical technique course grades than college SGPA or GPA.14,15 However, the SGPA is a much stronger predictor of performance on National Board Dental Examination (Parts I and II) scores than the PAT.1 Similarly, the AA has been shown to be predictive of the didactically oriented first year of dental school but not of performance in the clinically oriented fourth year of dental school.1
Admissions information has historically been used as a predictor of academic success in dental school. However, the admissions information can also be evaluated to help identify students that may be at risk for low academic performance, and the factors that predict success may not be factors that predict low performance.1,4 Sandow et al.1 evaluated 459 dental students over a four-year period and determined that the percentage of students who graduated with difficulty was 8.9 percent. These authors found that the students in academic trouble consumed a disproportionate amount of institutional resources and were shown to have lower undergraduate GPAs, lower DAT academic scores, and lower PMAT scores compared with students who graduated without difficulty. Likewise, the MCAT and college GPA have been shown to be predictive of medical students who either had a delayed graduation or were dismissed.16
At the University of California, San Francisco, we have constantly evaluated and reviewed admissions criteria in order to select a class that reflects our mission of excellence and scholarship. When a dental student is having academic difficulty, we will often evaluate the students college background to determine if a predictive pattern could be established. In that process, we have observed that some dental students having academic difficulty had college backgrounds that included a light academic load and often attended less academically rigorous colleges. We therefore sought to determine if the academic load and the rigor of the college attended, as well as more traditional admissions criteria, had predictive value in identifying underachieving dental students.
Our purpose was to determine if a group of underachieving dental students had scores on admissions criteria (GPA, SGPA, AA, PAT, college rigor, and academic load while in college) that differed from the scores of normally tracking dental students. Additionally, we sought to determine if students GPA at the end of the first year of dental school predicted their cumulative GPA at graduation.
| Methods |
|---|
|
|
|---|
Admissions data collected included science grade point average (SGPA), overall grade point average (GPA), two sub-scores of the DAT (the academic average, AA, and the Perceptual Ability Test, PAT), the ranking of the college attended, and the college academic load. The college ranking was an estimate of the academic rigor of the college attended, determined by using the 2006 online Princeton Review guide that lists the relative selectivity of colleges and universities. College rankings are listed on a scale from 60 to 100, and ranking is determined primarily by the average standardized test scores such as the Scholastic Aptitude Test (SAT) and GPA of the entering class. When a college does not submit profile information to the Princeton Review, the scores are automatically listed as a 60.
The college academic load was determined by reviewing the students college transcript and determining the percent of time the student completed a full academic load while in college. The college academic load was calculated by determining the percentage of quarters or semesters that the student completed equal to or more than sixteen quarter or twelve semester units. Summers were not counted.
Dental school GPA was determined by calculating the GPA at the end of the first year of dental school as well as a cumulative GPA at graduation.
Descriptive measures for all variables were prepared to assess general characteristics of the data and to observe variation among students performance, especially with respect to differences between the targeted groups and their corresponding admissions criteria and dental school outcomes. To identify if statistically significant performance differences existed between normally tracking and underachieving students, t-tests compared mean differences in admissions variables and dental school outcomes for both study groups.
Graphic and numeric measures of association were developed to understand general relationships among all of the variables and specifically to measure the level of correlation among various admissions criteria and dental school performance outcomes for normally tracking and underachieving students. Matrices with Pearsons multiple correlation coefficients were used to describe the individual bivariate associations among admissions criteria and first-year dental school GPA and graduation GPA. Scatter diagrams were also completed to visually illustrate the association between first-year GPA and GPA at graduation for the two groups under study.
Multiple regression models using SPSS software were developed to predict the individual effects of six admissions criteria upon dental school performance at first year and at graduation for normally tracking and underachieving students. Similarly, regression models for both student groups were developed to determine if dental school GPA at first year predicted graduating GPA. SPSS modeling included manual, stepwise, remove, backward, and forward entry procedures for independent variables, and all models were tested for multicollinearity.
| Results |
|---|
|
|
|---|
Descriptive measures and statistical t-test results presented in Table 1
and Table 2
, respectively, show that the normally tracking dental student group had statistically significantly higher scores than the underachieving student group for all admissions variables evaluated except PAT. While PAT differences were nearly statistically significant (p=0.07), all remaining comparisons were significant at least at p<0.05 with five of the eight (63 percent) total difference comparisons significant at p<0.001. Compared to underachieving dental students, the normally tracking dental student group had significantly higher first-year dental school GPA (3.43 ±0.28 vs. 2.47 ±0.21; p<0.001) and GPA at graduation (3.34 ± 0.27 vs. 2.55 ± 0.19; p<0.001).
|
|
|
|
|
|
| Discussion |
|---|
|
|
|---|
Regression modeling, Pearson correlation matrices, and graphic means of association indicate that, as measured by GPA, those students who performed well in their first year of dental school continued with similar performance in their graduating GPA and, conversely, students who underperform can expect comparably below average results. Explicitly, students first-year GPAs are strongly associated with their graduating GPAs, such that high/low achievers at the end of their first year of dental school tended to remain high/low performers at graduation.
Our finding of first-year dental school GPA being a significant predictor for the GPA at graduation is consistent with investigators who have found that early assessments within an educational program are often strong predictors of later performance.8,17 In a retrospective study of 352 medical students, Croen et al. found marginal performance on exams early in the first year were more predictive in identifying students at risk academically than college GPA, the MCAT, or both.17 Croen et al. attributed this finding to the similar environment of assessments and outcome measurements, which they felt afforded higher predictive validity. Our finding of first-year dental school GPA predicting graduating GPA for normally tracking and underperforming dental students (R2=0.88, 0.76) is consistent with Croen et al.s findings, as was our finding that admissions criteria were substantially weaker than first-year GPA in predicting graduating GPA. It has also been shown that students having difficulty in one first-year class tended to have difficulty in subsequent classes.8 In a study of 420 medical students, Hall and Bailey found that students who had difficulty in one first-year course tended to have academic problems in subsequent classes, and students who did well in early coursework tended to continue with high grades.8 Our finding of first-year dental school GPA predicting GPA at graduation better than admissions criteria tends to be supported by other investigators who have shown assessments and outcomes completed in a similar environment tend to afford stronger predictive validity.5,13
Our findings of generally weak correlations between admissions criteria and first-year GPA and GPA at graduation were consistent with other investigators who have compared average and underperforming dental students.1,4 We found that SGPA, AA, and the academic rating of the school were significantly correlated to first-year GPA in normally tracking students, while only overall college GPA was significantly correlated to first-year GPA in underachieving dental students. Our finding of only overall GPA being correlated to first-year GPA in underachieving students was similar to Sandow et al., who found that students dismissed for academic reasons had a much lower non-science GPA than average dental students.1 Additionally, in a study of dental students over a four-year period, Kramer and DeMarais found that students who withdrew for academic reasons tended to have lower academic qualifications.4
The correlations we determined between admissions criteria and dental school performance were generally lower than reported by several investigators1,6 but were similar to other investigators.3,12 Sandow et al.1 found admissions correlations to first-year GPA that averaged higher than we determined for SGPA (0.41 vs. 0.27) and AA (0.47 vs. 0.36), while our results averaged closer to Kramer, who determined correlations of 0.37 for SGPA, and Chamberlain et al., who found 0.37 for AA.3,12
While statistical significance affirms only that coefficient values are not zero due to random variation, the essence of the Pearson statistical measure is to communicate a degree of individual association among variables. None of the correlation coefficients for admissions variables approached 0.50, and most were well below 0.50, leading to our observation that admissions criteria are weak predictors for first- or fourth-year academic performance in dental school as measured by GPA. It is important to remember that correlations that are below .40 have limited predictive value, even if statistically significant.5 Our findings included a number of statistically significant correlations that, with the regression analysis, were shown not to be predictive in explaining the variance in first-year GPA or GPA at graduation. This need for caution in interpreting the predictive value of correlations has been emphasized by several investigators.3,5 Mitchell, in an example of correlations when evaluating admissions criteria, explained that with a perfect correlation of 1.0 all students in the top fifth of an applicant pool would graduate in the top fifth of the class.5 With a correlation of .40, only 28 percent of the students in the top fifth of the applicant pool would graduate in the top fifth of the class, while random selection would result in 20 percent of students in the top fifth of the class. Although a correlation may be statistically significant from zero, the correlation may not be predictive when tested with the more rigorous regression analysis. Kramer has stated that the degree of correlation does not necessarily indicate an acceptable level of predictive validity,3 which was certainly the situation when evaluating our data.
Our finding of the weak predictive value of admissions criteria to dental school performance is consistent with the general education literature.18–22 In a study of over 77,000 students enrolled within the University of California system over a four-year period (1996–99), it was shown that high school GPA, SAT I, SAT II, parental income, and parental education only predicted 19.4 percent (R2=0.194) of the variance in the first-year college GPA.19 The individual admissions criteria were even weaker, with high school GPA predicting 12.6 percent of the variance in first-year college GPA, SAT (8.4 percent), SAT writing (11 percent), and the SAT combined with SAT II (12.6 percent).19 This means over 80 percent of what explains GPA in the first year of college is not explained by the admissions criteria according to Zwick et al.19 The ability of the Graduate Record Examination (GRE) to predict grades in the first year of graduate school is also weak, showing correlations of between .15 and .32 depending on the subdiscipline.20,21 The correlations of standardized tests such as the Graduate Management Admission Test (GMAT) (.20 to .32 depending on subscale) or Law School Admission Test (LSAT) (.44) are similar to or slightly stronger than the correlations we determined.18,22
We have found that pre-admissions criteria traditionally used to evaluate applicants for dental school programs are not accurate predictors of dental school performance as measured by first-year or graduating GPA. We also argued that our results are similar to findings in the education literature in which the criteria often used to evaluate college applicants lack predictive validity for academic success in college. We have also argued that, in the dental admissions literature, the degree of validity often ascribed to Pearson product moment correlations demonstrating bivariate association is probably overstated.
These findings and arguments beg the question of what criteria should be used to review applicants for admission to dental school. While we do not have a definitive answer, the recent work by Poole et al. offers a promising direction.23 In a longitudinal study of 373 Canadian dental students, Poole et al. found that data gathered from structured interviews showed the applicants "conscientiousness" correlated significantly with all four years of dental school performance (r=0.24, 0.47, 0.32, 0.39). Although Poole et al. stated that the sample size was too small to produce a stable regression model, the correlation of r=0.47 for second-year dental school performance looks especially promising. We hope similar investigations of behavioral measures will be forthcoming and add to the criteria currently being used to evaluate applicants.
The academic load while in college was not found to help predict first-year GPA or graduating GPA. This finding is consistent with similar investigations. Staat and Yancey6 looked at the college credit hours in seventy-seven students at one dental school and found a slight negative correlation between college credit hours and first-year dental school performance.
| Conclusions |
|---|
|
|
|---|
| Footnotes |
|---|
| REFERENCES |
|---|
|
|
|---|
This article has been cited by other articles:
![]() |
J. Freudenthal and D. M. Bowen A Scholastic Appeals Process for Dental Hygiene Student Remediation and Retention J Dent Educ., March 1, 2010; 74(3): 268 - 274. [Abstract] [Full Text] [PDF] |
||||
![]() |
N. Lopez, K. Self, and J. Karnitz Developing a Tool for Systematic Inclusion of Non-Academic Factors in Dental School Admissions: Towards Building Diversity in the Dental Workforce J Dent Educ., December 1, 2009; 73(12): 1347 - 1352. [Abstract] [Full Text] [PDF] |
||||
![]() |
L. R. Kogan, S. M. Stewart, R. Schoenfeld-Tacher, and J. M. Janke Correlations between Pre-Veterinary Course Requirements and Academic Performance in the Veterinary Curriculum: Implications for Admissions J Vet Med Educ, June 1, 2009; 36(2): 158 - 165. [Abstract] [Full Text] [PDF] |
||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |