Table 2

Articles that met inclusion criteria

First Author, YearInstrumentDesignSubjectsResultsLimitationsReliabilityValidity
Arand, 1984Watson- Glaser Critical Thinking Appraisal (WGCTA)Longitudinal study to determine problem- solving skills of entering students and determine if exposure to problem-solving curriculum improved problem-solving, as measured by WGCTA; also examined relationships of grades in prerequisite courses, selectivity of school at which courses were taken, learning style, and major to problem- solving81 physical therapy students at one institutionNo significant predictors of problem-solvingHeterogeneous subjects; one institution; WGCTA used to evaluate problem- solvingNone reportedNone reported
Beck, 1999California Critical Thinking Skills Test (CCTST)Longitudinal study to assess change in CCTST from beginning to end of first year44 1st year nursing students at one institutionStudents showed significant gains in CCTST from pre- to posttest; most of gain was in medium- scoring groupSelf-selected sample; small sample sizeNone reportedNone reported
Behrens, 1996WGCTACross-sectional and longitudinal study evaluating WGCTA scores at program start; correlation with 1st semester GPA; change in WGCTA from start to end of program for one cohort109 full-time nursing students in certificate program in 3 consecutive cohorts at one institutionSignificant correlations be- tween WGCTA and GPA; no significant change in WGCTA over timeOne institution; small sample size for longitudinal aspect (n=17)Test-retest reliability (r=0.42)Predictive validity (r=0.51-0.59)
Bixler, 2015CCTSTPretest-posttest with one-month small group concept map- ping as intervention27 4th year medical studentsNo significant change in CCTSTSmall pilot study; brief intervention; no control groupNone reportedNone reported
California Critical Thinking Disposition Inventory (CCTDI)Pretest onlyCCTDI did not correlate with any other measureNo posttestNone reportedNone reported
Bondy, 2001CCTSTCross-sectional study of psychometric properties of CCTST and CCTDI; two-week retest of subset of subjects320 undergraduate students; retest on 156 who were nursing studentsCCTST demonstrated insufficient reliabilityOne institutionAlpha=0.68 (1st sample), 0.61 (2nd sample); stability estimates=0.73Construct validity: KMO=0.67 (mediocre); concurrent validity: correlation between CCTST and WGCTA r=0.43
CCTDICCTDI demonstrated good reliabilityOne institutionAlpha=0.89 (1st sample) and 0.90 (2nd sample); good reliabilityKMO=0.84 (meritorious)
Daiek, 1993CCTSTCorrelational study to identify CCTST skills and demographic factors that may impact first semester success59 disadvantaged students entering a college program at one institutionLow to moderate but significant positive correlations between overall CCTST and first semester GPA (r=0.29) and interpretation subscale and GPA (r=0.35)Small sample size; one institution; highly specific subject poolNone reportedNone reported
Denny, 2012CCTSTCorrelational study to examine relationship among participation in a research program, CCTST, and academic performance45 undergraduate students at one institution who participated in research programsStudents with 4 or more semesters of undergraduate research experience had higher CCTST scores than those with less research experienceOne institution; small sample size; heterogeneous research experienceCronbach’s alpha=0.97None reported
Facione, 1994CCTSTPretest-posttest and case-control study to assess reliability and validity of CCTST1196 college students at one institution; cases were students enrolled in courses that fulfilled a critical thinking requirementCCTST showed adequate reliability and validity; significant gains in CCTST scores for cases but not controlsNo predictive validity assessmentKuder-Richardson=0.68 to 0.69Concurrent: significant correlation with GPA and SAT; construct: cases but not controls increased scores
Fero, 2010CCTSTQuasi-experimental crossover design assessing relationship between CCTST and simulation-based performance36 nursing studentsSignificant relationship between problem recognition subscale and overall CCTSTSmall convenience sample; possible selection bias; possible differences between simulation performance and actual situationsNone reportedNone reported
CCTDISignificant relationship between overall performance on simulations and CCTDINone reportedNone reported
Frye, 1999WGCTACross-sectional and longitudinal study to determine correlation between WGCTA and National Council Licensure Examination (NCLEX), whether WCGTA was higher for 4th year than 1st year students, and whether it changed over time132 1st and 77 4th year nursing students at one institution; 27 students for longitudinal studyStatistically significant differences in WGCTA between 1st and 4th year groups; no significant change in WGCTASmall convenience sample from one institution; few males; insufficient statistical data reportedNone reportedPositive but unspecified correlation of 2 WGCTA subtests with NCLEX success; higher scores for 4th year than 1st year students
Gadzella, 2003WGCTALongitudinal design to assess validity and reliability of WGCTA for students majoring in education135 undergraduate education majors enrolled in an educational psychology courseWGCTA was a reliable and valid measure of critical thinking in education majorsSmall sample size precluded factor analysisCronbach’s alpha=0.86 Split-test reliability coefficient=0.75Concurrent validity: correlation with course grades (r=0.42, p<0.01)
Gorton, 2014CCTSTCorrelational study to determine relationship between CCTST and clinical judgmentConvenience sample (size not specified) of nurse practitioner students within one year of master’s degree completionNo relationship between CCTST and clinical judgmentSample size not stated; comparison measures lacking detail; outlier not discussedCronbach’s alpha=0.809None reported
Huhn, 2011Health Sciences Reasoning Test (HSRT)Parallel group study to determine construct validity of HSRT by comparing scores of expert and novice physical therapists (PT)79 1st year PT students and 73 experts in PT (based on clinical specialty certification)Experts had significantly higher HSRT scores overall and on two subscalesExperts were not monitored while taking HSRTKR-20=0.54 for analysis and 0.52 for inference subscalesConstruct validity based on higher HSRT scores for experts than novices
Ingle, 2007CCTSTCorrelational study to identify predictors of CCTST283 community college and university students at two institutionsMetacognitive self-regulation and elaboration were strongest predictors of CCTST; no other significant predictorsNon-experimental design; convenience sample of mostly femalesNone reportedNone reported
Kidd, 2003CCTSTRetrospective study to determine if CCTDI, CCTST, and other measures predicted GPA139 pharmacy students in 3 successive cohorts at one institutionCCTST was a significant predictor of pharmacy GPA and clerkship GPAAssessment of performance in pharmacy practice setting not standardized; convenience sample from one institutionNone reportedNone reported
CCTDICCTDI was a significant predictor of pharmacy GPANone reportedNone reported
Kokinda, 1989WGCTACorrelational study to determine if there was a relationship between WGCTA and academic achievement49 nursing students at one institutionSignificant positive correlation between WGCTA and academic achievementSmall sample size; one institutionNone reportedNone reported
Liu, 2016HEIghtenCross-sectional study to establish validity evidence for the 5 forms of the instrument3036 college students from 35 two- and four-year institutionsUnidimensional factor structure for three forms; good reliability for three forms; students with higher high school and college GPAs had higher critical thinking scores; seniors scored significantly higher than freshmenMotivation was not high for approximately 30% of test takers; those with a financial incentive had highest scoresAdequate total score and subscale reliability (alpha >0.70) for all but two forms; adequate internal structurePositive correlation with other academic indicators; higher scores for seniors than freshmen
Lobb, 2006WGCTACorrelational study to determine ability of WGCTA and other measures to predict 1st semester, 1st year GPA405 1st year pharmacy students over 6 consecutive years at one institutionWGCTA did not contribute significantly to prediction of academic performanceStudy limited to one institutionNone reportedNone reported
May, 1999CCTSTCross-sectional study to evaluate critical thinking skills and relationship between CCTST and clinical competence143 4th year nursing students at one institutionNo significant relationship between CCTST and clinical competenceCCTST and CCTDI not given in context of the curriculum and not taken seriouslyNone reportedNone reported
CCTDICronbach’s alpha=0.88; subscales ranged from 0.55 to 0.76None reported
Pardamean, 2007HSRTCross-sectional study to measure change in HSRT scores98 1st, 2nd, and 3rd year dental students at one institutionNo significant differences in HSRT across 3 years of dental studentsOne institution; cross-sectional design with intent to measure change; small sample and self-selected subjects; HSRT is a context-free test of critical thinkingNone reportedNone reported
Profetto- McGrath, 2003CCTSTCross-sectional study of CCTST and CCTDI228 nursing students in all 4 years at one institutionNo significant differences in CCTST among the 4 years; significant relationship between CCTST and CCTDIOne institution; 35% self-selected participationNone reportedNone reported
CCTDINo significant differences in CCTDI among the 4 yearsNone reportedNone reported
Robertson, 2001WGCTACorrelational study of relationship among WGCTA, personality style, and OSCE performance130 senior medical students at one institution in 3 successive cohortsNo relationship among WGCTA, personality style, and OSCE performanceSelf-selected subjects; timing of instruments; one institutionNone reportedNone reported
Ruland, 1999WGCTAPretest-posttest study of relationship between classroom environmental factors and growth in critical thinking342 1st year college students at one institutionClimate assessment scores predicted increase in critical thinking as measured by WGCTANon-experimental design; one institution; no control for non-classroom influencesNone reportedNone reported
Scott, 1998WGCTALongitudinal study to evaluate critical thinking skills and correlation of critical thinking with clerkship performance68 to 82 1st and 3rd year medical students at one institutionSignificant increase in mean WGCTA scores; weak correlation between WGCTA and clerkship performanceOne institution; clerkship evaluation measures unreliableNone reportedPredictive validity: weak correlation with clerkship scores (r=0.08 to 0.24)
Stone, 2001CCTSTCross-sectional study to examine validity evidence for CCTDI and CCTST in nursing students; included SAT, GPA238 (CCTDI) and 184 (CCTST) nursing students at one institutionSignificant correlation between total CCTST and SAT math, SAT verbal, GPAOne institution; not all subjects completed both measuresCoefficient alpha for total CCTST=0.61; low reliability for subscalesPredictive validity: low correlation with GPA; moderate correlation with SAT math
CCTDITraits measured by CCTDI did not relate to CCTSTCoefficient alpha for total CCTDI=0.89Predictive validity: very low correlations with GPA and SAT
Thompson, 1999CCTSTLongitudinal study to examine correlation of age, GPA, gender, and ethnicity with CCTST38 nursing students at one institutionTotal CCTST increased significantly over the course of four yearsPretest was not administered at beginning of curriculum; small sample; one institutionNone reportedNone reported
CCTDITotal CCTDI increased significantly over 4 yearsNone reportedNone reported
Tsai, 2014CCTSTCorrelational study to assess validity of assessing critical thinking on Dental Admission Test (DAT); examined relationship between CCTST and NBDE Part I performance439 3rd and 4th year dental students from 10 schoolsCCTST did not predict student performance or add to predictive power of DATDAT and CCTST data collected years apart; one-shot critical thinking assessmentNone reportedNone reported
Valdes- Corbeil, 2005WGCTACausal comparative study to determine effect of a test preparation program on WGCTA scores700 undergraduate teacher education program applicants at two institutionsWGCTA scores of subjects who participated in test preparation program were not significantly different from those who did not participateSelf-selected groups; cohorts not equivalent; ex post facto designNone reportedNone reported
Vaughan- Wrobel, 1997WGCTALongitudinal study to assess WGCTA differences across classes and over time391 nursing students from 3 consecutive cohorts at one institutionNo significant differences in WGCTA from entry to end of 3rd and 4th yearsOne institution; baseline assessment timing (summer before 3rd year)None reportedNone reported
Vendrely, 2005CCTSTLongitudinal study to determine CCTST changes as result of curriculum completion42 graduate physical therapy students at one institutionNo significant difference in CCTST from beginning to end of 27-month curriculumSample size; one institution; timing of posttest (last week of program)None reportedNone reported
Vogel, 2009WGCTAPretest-posttest with 20-month curriculum as intervention50 physical therapy (PT) and occupational therapy (OT) studentsSignificant increase in WGCTA for OT but not PT students over timeSmall sample size; differences in timing of critical thinking instruction between the two programsNone reportedNone reported
Wagner, 2006Wagner Assessment Test (WAT)Cross-sectional study to evaluate psychometrics of new critical thinking assessment similar to WGCTA but with more response alternatives407 undergraduates at one institutionWAT showed more measurement precision than WGCTANo consideration of gender or ethnic bias; criterionrelated validity not well establishedAlpha=0.93 for 80 item and 0.90 for 40 item versionsConvergent validity: r=0.69
Williams, 2002Williams Critical Thinking Assessment (CTA)Cross-sectional study to assess psychometrics of new critical thinking instrument and validate definition of critical thinking1461 1st year students at one institutionWilliams CTA compared favorably with other critical thinking skills instruments; good reliability for 6 domainsOne institution; homogeneous population; instrument reliability somewhat lowCronbach’s alpha=0.80-0.90Extensive factor analysis on 3 subinstruments; good construct and divergent validity
Williams, 2003CCTSTPretest-posttest study to validate use of CCTST and CCTDI to predict initial clinical performance207 1st yeardental hygiene students from 7 institutionsCCTST was a more consistent predictor of student clinical performance than age, college credits, or GPAFaculty rating scale was subjective; subject loss and missing data reduced sample size for several analysesTest-retest reliability=0.63Strong predictive validity
CCTDICCTDI did not predict clinical reasoning, faculty-rated performance, or acquired knowledgeTest-retest reliability=0.70Low predictive validity
Williams, 2006CCTSTPretest-posttest study to determine predictive validity of CCTST and CCTDI for National Board Dental Hygiene Examination (NBDHE)76 1st year dental hygiene students from 3 programsCCTST predicted performance on NBDHESmall sample sizeTest-retest reliability=0.61Moderate predictive validity (partial r-squared= 0.18)
CCTDICCTDI did not predict performance on NBDHETest-retest reliability=0.793Low predictive validity (partial r-squared= 0.000)
Zettergren, 2004CCTSTCross-sectional study assessing critical thinking for 3 cohorts at one institution200 3rd, 4th, and 5th year PT students at one institutionSignificant differences in CCTST between 3rd and 5th year and between 4th and 5th year studentsOne institution; small 5th year sample; authors reported significant increase in CCTST, but study was cross-sectionalNone reportedNone reported
  • Note: See reference list for full citations of articles. Data reported in Design column are as reported in the referenced study (if reported). Data in Reliability and Validity columns are as reported for this study.