|
|
||||||||
Perspectives |
Key words: teaching effectiveness, faculty development, dental education, continuing education, evidence-based practice
| Abstract |
|---|
|
|
|---|
"You cannot teach a man anything; you can only help him find it within himself."—Galileo Galilei2"The most extraordinary thing about a really good teacher is that he or she transcends accepted educational methods."—Margaret Meade2,3
"The mediocre teacher tells. The good teacher explains. The superior teacher demonstrates. The great teacher inspires."—William Arthur Ward2,4 (One of Americas most quoted writers of the twentieth century, Ward was a university administrator, college teacher, and high-level lay leader in the Methodist Church, who held administrative positions with the Red Cross and Boy Scouts, contributed regularly to Readers Digest, Phi Delta Kappan, and other periodicals, and wrote a daily column that appeared in more than 200 U.S. newspapers in the 1950s and 1960s.)
"Those who educate our youth are more honored than those who produce them; for the former only gave them life, and the latter the art of living well."—Aristotle2,5
"A teacher who is attempting to teach without inspiring the pupil with a desire to learn is hammering on a cold iron."—Horace Mann2,6 (The most famous educator in nineteenth-century America, Mann is known as the father of American public education. As secretary of education in Massachusetts in the 1830s and 40s, he established the first public school system in the United States and supervised the development of fifty school districts and the first teacher training academies. After serving as a U.S. senator, he was president of Antioch College and hired the first female faculty members at a U.S. college. His closing remarks at the 1859 Antioch commencement shortly before his death—"Be ashamed to die until you have won some victory for humanity"—became the school motto and a mainstay of commencement ceremonies.)
"In a completely rational society, the best of us would aspire to be teachers and the rest of us would have to settle for something less, because passing civilization along from one generation to the next ought to be the highest honor and highest responsibility anyone could have."—Lee Iacocca2,7
"It is in fact nothing short of a miracle that the modern methods of instruction have not yet completely strangled the holy curiosity of the inquiring mind."—Albert Einstein2,8
"The most admirable of reforms cannot but fall short in practice if teachers of sufficient quality are not available in sufficient quantity. Generally speaking, the more we try to improve our schools, the heavier the teachers task becomes; and the better our teaching methods, the more difficult they are to apply."—Jean Piaget9
As noted by Piaget, the success of educational reforms ultimately lies with individual instructors and their capacity, individually and collectively, to implement "new ways of doing things" during their day-to-day, hour-by-hour, moment-by-moment interactions with students. Dental educators are navigating through an era when numerous fundamental changes in curriculum format and teaching/learning methods have been proposed. Academic dentists and members of the practice community have been hearing, for more than a decade, that "our educational system is in trouble" and that the "profession has lost its vision and may be wavering in the achievement of its goals."10 These concerns are articulated in the 2001 Future of Dentistry report by the American Dental Association. This report asserts that the dental professions capacity to promote the oral health of the public depends on achievement of these goals: integration of emerging diagnostic, therapeutic, and communication technologies into practice; understanding and responding to evolving consumer needs and expectations; providing an adequate supply of well-trained dental educators and dental students; re-energizing and maintaining the research focus within the profession; and developing strategies to serve the needs of members of our society who do not have access to care.11 Pyle et al., writing for the American Dental Education Association (ADEA) Commission on Change and Innovation in Dental Education, argued that "there is a compelling need for rethinking the approach to dental education in the United States" and identified three issues that frame the case for change: the financial environment within higher education that makes dental school "expensive for universities to operate and tuition-intensive for students, producing high debt levels that limit access to education and restrict career choices; the professions apparent loss of vision for taking care of the oral health needs of all components of society and the resultant potential for marginalization of dentistry as a specialized health care service available only to the affluent; and the nature of dental school education itself, which has been described as convoluted, expensive, and often deeply dissatisfying to our consumers—the students."12
Figure 1
displays the dental education reform agenda, consisting of a bakers dozen list of modifications to the standard ways of doing things in U.S. dental schools that have been proposed many times in many ways by many individuals and groups.13–15 If implemented, these reforms will require faculty to make substantial modifications in their approaches to teaching. As noted in a companion article by Licari in this issue of the Journal of Dental Education, provision of faculty development related to teaching and assessment strategies is widely perceived to be the essential ingredient in efforts to introduce new curricular approaches and modify the educational environment in academic dentistry.16 Indeed, analyses of the outcomes of efforts to revise curricula in the health professions, almost always in the direction of the reform agenda in Figure 1
, have identified the availability and effectiveness of faculty development as one of the key predictors of the success or failure of reform initiatives.17–20
|
The overall purpose of this article is to provide information and insights about faculty development that may be useful to dental schools in designing professional growth opportunities for their faculty. Emphasis is placed on the critical role of faculty development in preparing teachers for taking on new roles and acquiring new instructional skills that may be needed to function effectively in implementing future dental school curricula.
| What Is Faculty Development? |
|---|
|
|
|---|
Faculty development has been described in various ways in the literature on cultivation of the professional skills of the academician, but consistent themes are evident. In one of the earliest works on the topic, Centra described faculty development as "the broad range of activities that institutions use to renew or assist faculty in their roles" and includes initiatives designed to improve the performance of faculty in teaching, research, and administration.22 Bland et al. described faculty development as a "planned program to prepare institutions and faculty members for their academic roles including teaching, research, administration, writing/scholarship, and career management."23 Bligh proposed that the goals of faculty development are to improve practice in teaching, research, and institutional service and also to "manage change by enhancing individual strengths and abilities as well as organizational capacities and the overall culture."24 Bligh noted that faculty development programs are tangible indicators of the institutions inner faith in their academic workforce. ONeill and Taylor observed that extensive faculty development programs in dental education characterize campus cultures that value teaching and that "faculty development is needed at all levels of faculty life, from novice instructor through the administrator to address the various levels in the educational enterprise."25 Hand used a panel of dental educators in a Delphi process26 to identify the competencies necessary for three dental faculty categories: clinical teachers (who focus primarily on instructional and curricular roles), clinical scholars (classic "triple-threat" faculty who pursue teaching, research, and service), and research-intensive scholars (who primarily focus on scholarship and training of future scholars) and proposed that these "competencies for effective dental faculty" be used to structure faculty development programs that will allow individuals, especially those new to academia, to attain these essential skills.27
Finally, Steinart et al. proposed that it is reasonable for institutions to expect that faculty development will "result in improved teaching performance and better outcomes for students."28 According to Steinart et al., examples of such improvements include development and use of new teaching methods and evaluation techniques, enhanced processes for analyzing, planning, and implementing curricula, new perceptions about factors that influence relationships between students and teachers, and increased commitment to educational scholarship—investigating teaching and learning processes.
| How Is Faculty Development Accomplished? |
|---|
|
|
|---|
Centra recommended that enhancement of faculty instructional skills and capacity to guide students learning is primarily accomplished through a formalized professional development system, linked to the institutions process for making promotion and tenure and merit pay decisions, in which faculty and their chairs, with support from senior colleagues as mentors/role models, develop and continually update career growth goals and plans that are formally negotiated as a component of the faculty members annual performance evaluation.22 Centras concept of formal career planning with stipulated professional growth activities is consistent with Ullian and Stritters organizational strategies for promoting teaching effectiveness. Our review of the literature indicates that virtually all descriptions of faculty development strategies and opportunities fit within this hierarchy.
| Why Is Faculty Development Particularly Important in Dental Education? |
|---|
|
|
|---|
As has been widely discussed, academic dentistry is graying rapidly and continues to struggle to attract younger dental professionals into the educational arm of the profession. For the academic year 2003–04, 50 percent of dental school deans reported that faculty recruitment and retention were significant problems, and more than half of dental school deans indicated that they expected filling vacant positions to become more difficult in the future.30 In 2004–05, 1,039 faculty (almost 10 percent of the dental faculty workforce) left dental education, with 36 percent of these departing individuals entering into private dental practice. Younger faculty members (less than forty-five years of age) of lower academic rank were predominant among those who left dental schools to enter private practice.31 In academic dentistry, the loss of substantial numbers of junior faculty is a critical concern because the strength of educational programs rests on teachers and researchers who plan lifelong academic careers and have the time over the span of their careers to acquire the competencies and academic credentials to become valuable contributors to the overall mission of the parent institution. In 2004–05, dental schools recruited 1,150 new faculty members.31 Over the past decade, the numbers of faculty who leave dental education have been replaced by almost equivalent numbers of individuals who are "cannibalized" (hired away) from other dental schools or are completely new to the academic environment. Many of the new recruits are older individuals in the fifty-five to sixty age range who are ending their private practice or military/public health dentistry careers. In most cases, the older, "second career" recruits into dental education have little knowledge of educational theory and teaching practices and have limited, if any, experience with policies, procedures, and general expectations of the academic environment. As described by Bertolami, dental education is more and more "hoping that there will continue to be a supply of retiring practitioners and military dentists in the fifty-five to sixty age range who desire to serve as clinical supervisors for a few years prior to complete retirement."32
This situation is a double-edged sword for academic dentistry. On the one hand, there is conviction among some dental educators, and increasingly from external influences (see, for example, the New York Times article of October 11, 2007, titled "Boom Times for U.S. Dentistry, But Not for Americans Teeth"33), that the dental school curriculum needs reform; an academic culture that values the process of scientific inquiry and the scientific basis for patient care needs to be re-established; and graduates need to be instilled with a sense of social responsibility. On the other hand, the ranks of academe are increasingly populated by older but relatively inexperienced teachers moving in from the practice environment and bringing with them memories of the "way we were taught" in the 1970s and perhaps not eager to find themselves in the middle of a major upheaval in teaching methodology as they start what amounts to a transitional period between their primary career and retirement.
As Winston Churchill observed, "Any message of change must first be heard."34 Are the demographics of dental academe in the early twenty-first century such that reform messages will be heard? Churchill also famously said, "The winds of change are blowing and we lean into them with equal measures of anticipation and dread,"34 to reflect the conflicting emotions associated with even the anticipation of a future change in the way things are done.
| What Happens When Faculty Development Does Not Accompany Educational Reform? |
|---|
|
|
|---|
Analysis of the adoption, or lack thereof, of four frequently advocated new ways of doing business is instructive. These four reforms, which have been extensively endorsed throughout all components of health professions education, not only dental schools, are: 1) problem-based learning (PBL); 2) applications of information technology such as web-based (online) learning and digital versions of textbooks and other educational materials collectively known as e-learning or e-curriculum; 3) creation of thematically or topically integrated curricula with courses that are team taught across traditional academic boundaries; and 4) incorporation of the principles and techniques associated with evidence-based practice, which includes emphasis on helping students acquire the intellectual skills associated with critical appraisal.
Numerous reviews of the level of adoption of these educational innovations and barriers to incorporation into routine curriculum operation point to one key determinant of the fate of these initiatives: the presence, extent, and quality of faculty development efforts.13,16–19 For example, investigations of the effectiveness of strategies to help faculty acquire skills necessary for implementing PBL and instilling the confidence to make the leap necessary to actually use PBL in courses have demonstrated the need for active, hands-on learning with extensive practice opportunities and ample shadowing of, and mentoring by, experienced PBL facilitators.36–39 Yet reviews of learning outcomes that can be associated with PBL and the extent of implementation of this educational model point to failures in faculty development as one of the pivotal reasons for poor outcomes, negative attitudes among both students and faculty, and even program abandonment.40–43 Dalrymple et al. have described a longitudinal series of intensive, experiential PBL tutor training workshops that model best evidence teaching/learning strategies (as reviewed below) for faculty development and are consistent with recommendations that have emerged from the PBL literature related to development of faculty for new and different roles.44,45
Electronic curriculum, or e-curriculum, refers to various forms of computer-based learning including providing students with educational materials on DVD, online courses, electronic mechanisms to search the literature, email, and various applications of information technology including providing laptops to students, use of computer-based simulations in preclinical labs, podcasting, videoconferencing, use of PDAs, use of the Internet in the classroom during educational activities, and many other applications. In 2002–03, Kassebaum et al. found high levels of interest among dental educators in e-curriculum: 86 percent of North American dental schools reported that they had already expanded use of information technology (IT) in their curricula, and 82 percent of schools desired to increase IT even further during the next three years.13 Hendricson et al. reported that virtually all U.S. and Canadian dental schools had made substantial financial investments in e-curriculum infrastructure and resources, running into millions of dollars at individual health science centers.46 These investigators also reported that sixteen North American dental schools in academic year 2002–03 (approximately 25 percent of all dental schools) required their students to purchase or lease laptops as a matriculation requirement and a number of these schools had contracts with a commercial vendor to supply a digital version of all curriculum materials and textbooks bundled with the required laptops.46,47
In spite of efforts by dental schools over the past decade to provide IT resources for students and faculty, only a handful of studies have been conducted to determine the extent to which the numerous forms of e-learning have been incorporated into the curriculum by faculty and the extent to which course design and teaching/learning strategies have changed as a consequence of these initiatives. Hendricson et al. examined e-curriculum utilization in two studies published in 2004 and 2006. The first project assessed e-curriculum implementation at all U.S. and Canadian dental schools,46 and the second analyzed perceptions of students and faculty at U.S. dental schools with high levels of investment and infrastructure for information technology.47 The initial study of all schools revealed extensive IT advocacy, but limited incorporation of e-curriculum capacities into the mainstream, core curriculum by faculty.
A sample of 800 students, at all academic levels, at fourteen U.S. dental schools with a major IT commitment provided a mixed assessment of the value and practical utility of mandatory laptop programs, curricular support software, and web-based learning. Students reported there was minimal necessity to use laptops, associated software, or the Internet in order to perform well in their courses, and few students reported changes in study methods because most of their instructors had not modified their courses to incorporate laptop or Internet-related activities. The benefits perceived by students were primarily related to enhanced email communication with classmates and instructors, convenient access to the Internet, and the ability to receive their teachers PowerPoints. Less than one-third of students agreed that the value of the laptop and associated software was equal to the added tuition costs. Responses by 350 faculty members at these fourteen schools revealed a similar pattern: few had made extensive use of the capacities of the laptop and bundled software, the Internet, or web-based course management systems such as Blackboard, other than as repositories for standard course materials such as the syllabus and Power-Point files and as a means to send announcements to students.48 Academic administrators, students, and faculty all identified lack of faculty development as a significant component in the pattern of slow and hesitant adoption of information technology, a finding echoed by Andrews and Demps in a similar study of IT utilization completed by more than 1,000 course directors at North American dental schools in the same time period.46–49
Failure to meaningfully integrate information technology into educational programs is not unique to dental education. Zemsky and Massy, in a study titled "Thwarted Innovation: What Happened to E-Learning and Why," investigated e-learning (defined similarly to e-curriculum) at six colleges and universities that had made major investments in information technology, ranging from small liberal arts schools such as Hamilton College in New York with less than 2,000 students to major public universities such as the University of Texas at Austin and Michigan State University with many thousands of students.50 This study explored three assumptions that have driven efforts to reform higher education through the application of information technology. Below, the investigators conclusions about the accuracy of these assumptions follow the statement of assumption, followed by quotes from their conclusions section:
Zemsky and Massy concluded, "E-learning will only become pervasive when faculty change how they teach—not before."50
The creation of thematically or topically integrated curricula with courses that are team taught across traditional academic boundaries has been repeatedly advocated by curriculum reformers as one of the strategies to build a more meaningful and structurally coherent learning experience for students and also as a mechanism to promote students capacity to blend together the various pieces of the basic, behavioral, and clinical sciences as they acquire foundational knowledge.14,15,51–54 Yet Kassebaum et al. found that only a small percentage of North American dental schools (7 percent) followed a curriculum that could by any stretch of the definition be characterized as cross-disciplinary; instead, the vast majority of dental schools followed a traditional "smokestack" or silo curriculum, in which each department conducted its own hierarchy of courses throughout the years of dental school with little interaction with other departments.13 The majority of participants in the Kassebaum et al. study, who were primarily academic affairs/curriculum deans, indicated that efforts had been made at their schools to establish a more integrated approach to the curriculum and remove some of the silos, but expressed frustration with both the process of trying to accomplish this reform and the outcomes: the process is "slow and difficult," "departments remain territorial," "change is a slow and humbling process," "we tried but the faculty would have nothing to do with it," and "there was much planning and debate, but ultimately, it led to no changes of any consequence."13 Strategies proposed by curriculum specialists to overcome resistance to "breaking out of silos" focus on the need to "open faculty eyes to new ideas" and build both competence and confidence through faculty development.55–57 The top recommendation among participants in the Kassebaum et al. study was to enhance the availability of resources and time for faculty development to help course directors learn the skills needed to implement curricular reforms that move away from the "sage on the stage" teaching model.13
The fourth item on the bakers dozen dental education reform agenda is evidence-based practice, which has been proposed as a mechanism to accomplish several goals, including educating dental students to provide patient care that is supported by research evidence versus the historical "in my experience" approach58 and to instill an educational culture that values and promotes intellectual curiosity, based on the intertwined mental capacities of critical appraisal, self-directed learning, self-assessment, and reflection upon actions, decisions, and behaviors.32,59 Teaching and learning strategies associated with the development of critical appraisal and critical thinking skills have been identified and widely communicated to the academic community over the past twenty-five years.60–65 The philosophy and techniques of evidence-based practice (EBP) have also been extensively discussed in the literature and routinely appear in the core values and curricular philosophies of the accreditation standards of the major health professions education programs including the Commission on Dental Accreditation.66–68 Many health professions schools report efforts to introduce concepts that are common to both EBP and the processes for critical thinking to their students,13 yet reports have emerged that these educational experiences are not being readily transferred to the clinic or carried forth into practice after graduation.69,70
In 2004, for example, Coormarasamy and Khan conducted a comprehensive review of published research related to educational strategies to help medical students and residents (postgraduates) acquire critical skills associated with EBP such as question formulation, identification of information sources, literature searching, and writing critical appraisal summaries.71 Their objective was to evaluate the effects of what the authors termed "stand-alone" (primarily classroom-based) teaching strategies (described as journal clubs, lectures, and brief thirty- to sixty-minute workshops without other supportive learning experiences) versus clinically integrated teaching (defined as case analysis simulations, bedside teaching in the clinic, case conferences in the clinic based on recent patient encounters, literature-searching assignments related to patient symptoms, case presentations using EBP principles including an appraisal of research evidence pertinent to the patients condition, and EBP-driven chart audits to compare patient management to evidence available in the literature).
The investigators searched Medline, ERIC, the Cochrane Controlled Trials Register, the Cochrane Database of Systematic Reviews, the Database of Abstracts of Reviews (DARE), Best Evidence in Medical Education (BEME) reports, and Science Citation Index. A total of forty-two research reports were identified, but only twenty-three met standards for research design determined by the investigators. Eighteen studies measured the effects of stand-alone/ classroom training, and five assessed the outcomes of clinically integrated training. Of these twenty-three studies, four were randomized trials comparing two or more EBP training methods, and the remaining nineteen were non-randomized studies. Most of the latter group employed pre- and post-testing to measure changes in knowledge, critical appraisal skills, attitudes, confidence levels, and behavior (the latter were typically measured by self-reports of EBP use). Seventeen of the twenty-three studies assessed general EBP knowledge, and all investigations, whether classroom-based or clinically integrated, found enhancements in knowledge as a consequence of training. Nine of the studies measured acquisition of specific critical appraisal skills.
The authors concluded there was weak and inconsistent evidence that the stand-alone, classroom-based studies improved critical appraisal skills, but the clinically integrated studies all reported substantial enhancements in appraisal competency. Six of the studies assessed changes in attitudes and confidence levels. None of the stand-alone studies reported changes in attitudes; however, in the clinically integrated training, all studies reported improvement in student/resident attitudes and confidence in applying EBP techniques to patient care. Fourteen studies also assessed changes in actual behaviors, usually via self-report, although two studies employed chart audits and observation/feedback by supervising clinicians. Overall, little change in behavior was found among participants in the stand-alone, classroom-oriented training groups, while all five of the clinically integrated studies reported changes in behavior in desired directions, including changes in reading habits, selection of evidence sources, and capacity to access information resources, as well as documented changes in patient management strategies and use of clinical practice guidelines.
From these data, the authors concluded that stand-alone, classroom-oriented teaching and clinically integrated learning experiences are both effective in improving the knowledge base, but it is the clinically integrated teaching of EBP, embedded with actual patient care and realistic simulations, that is most likely to produce changes in skills, attitudes, and behavior. Notably, all four of the studies that employed a randomized controlled design produced outcomes that supported the effectiveness of clinically integrated learning experiences. Coormarasamy and Khan conclude, "It is important that teachers of critical literature appraisal and evidence-based medical practice consciously find ways of integrating and incorporating the teaching of critical appraisal into routine clinical practice"—a concept supported by the outcomes of a similar review by Werb and Matear,58 who discussed the importance of faculty development to provide instructors with the capacities to design and orchestrate clinically relevant learning experiences. In spite of the findings from systematic reviews of EBP teaching, reports persist that health professions educators still primarily restrict EBP training to classroom exercises rather than more authentic learning experiences in the context of ongoing patient care, and there have been numerous reports that the concepts and techniques of EBP are not being readily translated into clinical practice.72–75
In summary, all four of the core items on the education reform agenda (PBL, information technology, integrated, cross-disciplinary learning experiences, and evidence-based practice/critical appraisal skills) have struggled to find acceptance and routine use by faculty. As proposed by Coormarasamy and Khan, the barrier appears to be lack of awareness and/or lack of capacity among faculty to implement appropriate teaching/learning strategies. So, why not just "train them up" in these new techniques and "get on with it," as the old saying goes. If only it were that easy.
| Why Are Teaching Attitudes and Behaviors So Difficult to Change? |
|---|
|
|
|---|
|
According to Szulanski, innovations can become stuck at each of the four stages, but problems or even failure are most likely during the initial implementation stage (as highlighted in Figure 2
) for the following reasons: 1) poor communication between the inner circle advocates and the outer circle, who are expected to carry the load of implementing the innovation even though they were not involved in planning; 2) lack of motivation among the outer circle and/or personal lack of confidence in their capacity to implement the new approach, which often leads to defensiveness and passive-aggressive behavior;79 3) failure to prepare the outer circle for new tasks and roles; and 4) failure to provide encouragement and rewards for the outer circle, especially as they struggle with the predictable and often discouraging pitfalls of early implementation.80
Analysis of efforts to implement the bakers dozen reforms suggests that dental schools have experienced difficulties with the hand-off from the inner circle of faculty and administrators, who are enthusiastic about these reforms, to the frontline faculty in the trenches. All four of Szulanskis points of "stickiness" are directly or indirectly issues of faculty development. For example, Hendricson and Cohen14 identified four factors that mitigated against faculty adoption of PBL in dental schools: 1) the student-centered structure of PBL is alien to most dental school faculty, who had no experience with case-based and student-centered learning when they were in dental school; 2) department chairs have concerns about the resources needed to implement PBL and consequently are hesitant to commit their teachers to a curriculum format perceived to be faculty-intensive; 3) many dental school faculty are not comfortable with the "solve the mystery" process of PBL, which is essentially a diagnostic detective game, and have difficulty watching students debate the merits of various patient care approaches, which faculty may see as time-consuming and inefficient; and 4) faculty repeatedly ask to be shown the evidence that PBL is a sound methodology, but often do not receive convincing answers to rectify the perception that PBL is more work for the already overworked.
Critiques of the PBL movement concluded that adoption of this educational innovation would have been enhanced by three events that largely did not occur: 1) avoid the "build it and they will come" approach by providing meaningful training to faculty prior to implementation of this curriculum model; 2) establish that the technique is indeed worth the extra effort by conducting research that demonstrates that assumptions about educational advantages are true; and 3) once events 1 and 2 have occurred, implement a transformational approach to ensure a comprehensive implementation of the innovation throughout the curriculum versus relegating educational innovations to the category of supplements and add-ons that can be ignored or dismissed by both students and faculty if they so desire.37,44,81
The take-home message from this section is that orchestrating educational reform is a complex sociological, psychological, and organizational undertaking and that faculty development lies (or should lie) at the heart of the planning and implementation of the process employed to roll out the new way of doing things.
| What Outcomes Can Be Expected from Faculty Development? |
|---|
|
|
|---|
Appraisal of the Outcomes of Continuing Education Programs for Providers
A team of investigators led by Davis and Haynes has evaluated the effects of continuing education programs on professional practice and health care outcomes for more than twenty years, producing a series of landmark reports beginning in 1984 with an article in the Journal of the American Medical Association (JAMA) aptly titled "A Critical Appraisal of the Efficacy of Continuing Medical Education."89
The second review in this series was published in 1992 in JAMA and was titled "Evidence for the Effectiveness of CME: A Review of 50 Randomized Controlled Trials."90 The 1992 report caught the attention of the health professions education community and became one of the most widely cited educational articles of that decade. The sophisticated literature selection and appraisal techniques described in this seminal report and the analysis of the available data set the standard for subsequent reviews of outcomes evidence in health professions education. After applying rigorous criteria for research design, subject section, randomization, study implementation, and data analyses that led to the exclusion of hundreds of reports with insufficient research quality, Davis et al. identified fifty randomized controlled trials (RCTs) that assessed CE outcomes for knowledge, attitudes, skill acquisition, and changes in providers behaviors. Davis et al. found that CE interventions that relied exclusively on lecture-based methods had the capacity to improve participants fund of knowledge, primarily in the short term (immediately after course conclusion) as measured by pre- to post-testing, but were uniformly ineffective in promoting substantial changes in attitudes, skill levels, or confidence and rarely resulted in application of skills during actual patient care. In contrast, a subset of fourteen RCTs within the overall group of fifty studies produced different outcomes. These fourteen physician education programs employed a variety of active, learner-centered, and practice reinforcement techniques to augment lectures or were implemented largely without lectures. These techniques included completing self-assessments of practice behaviors or self-audits of patient records to identify current practice patterns and learning needs, using algorithms to solve clinical problems, applying group analysis to case scenarios, rehearsing the use of clinical practice guidelines in workshops, observing and critiquing the performance of other practitioners, receiving feedback from other providers during patient care simulations, doing problem solving to identify ways to reduce implementation barriers for practice guidelines, using debriefing exercises in which participants shared their opinions and concerns about new clinical techniques, debating the merits of practice recommendations, and distributing pocket cards, chart reminders, and posters that portrayed clinical practice recommendations. Significant changes in the desired direction in the patient care behaviors, attitudes, and confidence levels of participating physicians were found in all fourteen interventions that primarily employed active-learning strategies. Positive results for patient outcomes were also found in five of the nine studies that measured patient outcomes.90
In the years since this initial review of the evidence, the Davis group has periodically conducted other analyses either to update the literature as new research findings become available or to investigate other aspects of continuing education for physicians and occasionally other health care providers. Several of the principal reports appear in the references.91–95 The findings and conclusions from these reviews have not fundamentally differed from the outcomes of the 1992 JAMA report. The groups most recent review, published in 2005 in the Cochrane Database of Systemic Reviews (Issue 3), was titled "Continuing Education Meetings and Workshops: Effects on Professional Practice and Health Care Outcomes." This analysis reviewed the findings from thirty-two studies that employed randomized trials or well-designed quasi-experimental studies with a total of 2,995 health care providers from several different disciplines. These providers participated in a variety of planned professional development activities that occurred off-site from the practice setting including meetings, lectures, workshops, symposia, and weekend courses. Ten of the studies evaluated the effects of CE programs that used active-learning strategies similar to those described for the 1992 JAMA report by the Davis team; nineteen studies addressed outcomes from CE activities that used mixed lecture and interactive strategies; and seven studies evaluated exclusively lecture-based instruction. The investigators concluded that interactive programs that employed a mixture of primarily hands-on, active-learning strategies resulted in moderately large changes in professional practice; lectures alone, they found, were unlikely to change professional practice.95 The pattern of findings from the longitudinal series of reviews by the Davis group parallels the findings for training outcomes related to evidence-based practice by Coormarasamy and Kahn previously described.71
Appraisal of the Outcomes of Faculty Development Programs to Improve Teaching Effectiveness
The most comprehensive source of information about the effectiveness of faculty development initiatives to enhance teaching is the 2006 review by the Best Evidence in Medical Education Collaborative (BEME).28 The summary report (BEME Guide No. 8) appears in the June 2006 issue of Medical Teacher. This report is an outstanding synopsis of the available research and also provides useful guidelines for the design of faculty development programs as well as strategies to assess outcomes. The authors also included a summary of eight studies of faculty development outcomes that received the highest ratings for quality of research design from among the entire sample of studies that were critiqued for inclusion in this systematic review. I encourage all dental educators and others who are interested in professional development techniques to place this report in their "must read" folders.
The BEME is an ad hoc international organization of health professions educators who share a mission to move the education of care providers from "opinion-based education to evidence-based education."96 The BEME has produced a number of systematic reviews of the evidence related to various aspects of health professions education, with obviously a strong orientation to medical education, in an effort to inform, guide, and improve educational practices and also to build a culture among faculty and academic program managers that is accepting and supportive of best evidence approaches. The BEMEs search for evidence considered all forms of teaching enhancement strategies described by Centra22 and Ullian and Stritter29 above.
The BEME group used Kirkpatricks four-level hierarchy of educational outcomes (shown in Figure 3
) as the framework for classifying and analyzing findings from studies included in the data set.97 Kirkpatricks model has four levels of educational outcomes: 1) the learners reaction to the educational experience (opinions and perceptions); 2) indicators of learning, which include changes in attitudes, knowledge, and skills; 3) behavior, which includes changes in practice (i.e., modifications in methods); and 4) results, which refer to changes in the practices, policies, or infrastructure of the overall organization and/or changes in the practices/behavior of the students or colleagues of a faculty member who participated in a faculty development program.
|
All of the reports described a wide range of teaching/learning methods: no programs were completely lecture-based, and all reported some type of experiential learning component including observation of practice teaching assignments followed by feedback from other program participants or course faculty. Based on evidence reviews for EBP training and continuing education, the reliance on a blend of learning experiences appears to be indicative of best practices, but the diversity of methods influenced the assessment of the BEME data, in effect, producing an appraisal of the effectiveness of faculty development programs that used experiential learning. The majority of studies (forty-seven; 89 percent) used a quasi-experimental design, and six were RCTs. Among the quasi-experimental studies, thirty-one used pre- and post-testing to measure one or more outcomes. Data collection methods to assess outcomes varied widely, and most articles reported use of several measures of effectiveness. The most frequently used techniques were participants self-reports of post-program changes in knowledge, attitudes, and utilization of skills (fifty studies; 94 percent); end-of-program questionnaires to elicit participants opinions about program value and quality (forty-nine studies; 92 percent); direct observation of participants teaching performance by peers, educational specialists, or experienced faculty (sixteen studies; 30 percent); videotaping of teaching during or after the program for self-assessment and/or critique by others (fifteen studies; 28 percent); ratings of the participants teaching by students or residents (fifteen studies; 28 percent); and comparison of pre- and post-written tests to measure changes in knowledge, confidence, and attitudes about the topic of the program (apparently fifteen studies; 28 percent, although the BEME report is not completely clear about the number of studies that used pre- and post-program testing). Other outcome measures used less frequently included expert opinion about the participants teaching skills either during or after the program (e.g., critiques provided by educational specialists or experienced faculty), the scores of the participants students on examinations, and patients ratings.
The BEME report analyzed data using Kirkpatricks four-level outcomes hierarchy according to each program format (workshops, seminar series, short concentrated course, fellowship, and mixed method). The preponderance of available evidence indicated that all of the formats produced positive outcomes for reaction, learning, and behavior to the extent that these dimensions of the hierarchy were actually measured and given the nature of the assessment strategies. Outcomes pertinent to level four (results: changes in practices, policies, or infrastructure of organization or changes in the practices of students or colleagues of faculty development participants) were relatively sparse (nine studies reported level four outcomes), yet almost all were judged to be positive by the BEME analysts. As noted previously, all fifty-three studies employed a combination of teaching/learning methods with a decided tilt toward experiential learning, practice teaching, and self-, peer, and expert assessment. Consequently, these data reflect an assessment of the outcomes of experientially based faculty development that did not rely as heavily on lectures as the studies included in the EBP or CE reviews. In that context, the outcomes appear to be good news for those who endorse student-centered, experiential learning; however, the heavy reliance on participants self-reports to document changes in attitudes, knowledge, and behaviors should be noted in a cautionary sense (that is, consider the source). Approximately 40 percent of the studies reported outcome measures that included direct or videotaped observation of teacher performance or students/residents ratings of performance. Virtually all of these assessments of actual teachers behaviors were characterized as being positive, but many did not include a pre-training comparison to serve as a benchmark. Additionally, only eight of the fifty-three studies (the six RCTs and two of the quasi-experimental studies) employed a control group of faculty who did not undergo faculty development. If the Strength of Recommendation Taxonomy (SORT),98 which is widely used in the EBP literature, was applied to these data, the evidence and associated practice recommendations would probably grade out at the lowest level, C, although the evidence would score high for consistency. The SORT grades appear in Figure 4
.
|
Although these are desirable outcomes, evidence in the health professions literature that faculty development unequivocally does enhance teaching effectiveness is in short supply. Well-designed studies are needed to measure the outcomes of faculty development programs that incorporate the characteristics described in the next section. Implementation of such studies represents a substantial opportunity for dental educators to advance the understanding of strategies that have the capacity to improve teaching effectiveness and also provide the necessary support for curriculum reforms.
| What Does the Available Evidence Tell Us About the Design of Faculty Development Programs? |
|---|
|
|
|---|
| Final Thoughts: What About the Content of Faculty Development? |
|---|
|
|
|---|
One of the most influential books on educational planning in the twentieth century emphasized the importance of "getting the content of training right." This book was The Saber-Tooth Curriculum by Abner J. Peddiwell; it was published in 1939 but still remains a required reading in schools of education and in many business schools.106 Peddiwell tells the following story. During most of the Paleolithic era in Northern Europe, which extended from about 130,000 years ago when stone tools were developed to 10,000 years ago when agriculture was introduced, the young men of the pre-agrarian nomadic clans that roamed the continent were taught how to guard the perimeter of the camp at night with fire and noise to scare away saber-tooth tigers. According to Peddiwells story, the training included instruction on how to make large bonfires, how to build torches that would burn for long periods of time, how to space fire-wavers around the camp perimeter for maximum effect, and how to supplement torch waving with loud noises known to frighten saber-tooth tigers. In essence, the young men were participants in a critical professional development activity that was central to the survival of their parents and siblings and other families in the clan and which obviously addressed an important need. Due to the quality of the training, this technique for warding off saber-tooth tiger attacks worked well for several thousand years and became ingrained into the culture of the roving nomad tribes of that prehistoric era. However, over the next 10,000 years, an ice age slowly enveloped most of Europe. Saber-tooth tigers moved south to seek alternative food sources and a more hospitable climate; those that remained perished in the cold weather. Yet, among the nomadic clans, tiger scaring remained the core of the training for young men, even though there were fewer and fewer saber-tooths to scare away. Because of the traditional focus on fire-making, the elders of the clan failed to teach the younger generation more currently necessary skills such as finding alternative food sources, locating warmer places of shelter, and creating warmer clothing. The failure to transition the content of the training to cold weather survival skills had disastrous effects on these nomadic tribes.
Today, needs assessment and anticipatory planning are just as important as they were in the Paleolithic era 50,000 years ago. Let us use those tools to keep dental education relevant and effective for the future.
| Footnotes |
|---|
This article is one in a series of invited contributions by members of the dental and dental education community that have been commissioned by the ADEA Commission on Change and Innovation in Dental Education (CCI) to address the environment surrounding dental education and affecting the need for, or process of, curricular change. This article is authored by the ADEA CCI but does not necessarily reflect the views of ADEA or individual members of the ADEA CCI.
| REFERENCES |
|---|
|
|
|---|
This article has been cited by other articles:
![]() |
N. K. Haden, W. D. Hendricson, J. W. Killip, P. N. O'Neill, M. J. Reed, G. Weinstein, J. N. Williams, and R. W. Valachovic Developing dental faculty for the future: ADEA/AAL Institute for Teaching and Learning, 2006-09. J Dent Educ., November 1, 2009; 73(11): 1320 - 1335. [Abstract] [Full Text] [PDF] |
||||
![]() |
K. F. Novak What the ADEA CCI Series of Articles Means to Me: Reflections of a Mid-Career Dental Faculty Member J Dent Educ., February 1, 2009; 73(2): 166 - 171. [Abstract] [Full Text] [PDF] |
||||
![]() |
H. F. Thomas What the ADEA CCI Series of Articles Means to Me: Reflections of a Dental School Dean J Dent Educ., February 1, 2009; 73(2): 172 - 176. [Abstract] [Full Text] [PDF] |
||||
![]() |
R. S. Wilder, A. M. Iacopino, C. A. Feldman, J. Guthmiller, J. Linfante, S. Lavigne, and D. Paquette Periodontal-Systemic Disease Education in U.S. and Canadian Dental Schools J Dent Educ., January 1, 2009; 73(1): 38 - 52. [Abstract] [Full Text] [PDF] |
||||
![]() |
G. Crain Managing Change in Dental Education: Is There a Method to the Madness? J Dent Educ., October 1, 2008; 72(10): 1100 - 1113. [Abstract] [Full Text] [PDF] |
||||
![]() |
J. S. Wallace and T. D. Infante Outcomes Assessment of Dental Hygiene Clinical Teaching Workshops J Dent Educ., October 1, 2008; 72(10): 1169 - 1176. [Abstract] [Full Text] [PDF] |
||||
![]() |
N. K. Haden, W. Hendricson, R. R. Ranney, A. Vargas, L. Cardenas, W. Rose, R. Ross, and E. Funk The Quality of Dental Faculty Work-Life: Report on the 2007 Dental School Faculty Work Environment Survey J Dent Educ., May 1, 2008; 72(5): 514 - 531. [Abstract] [Full Text] [PDF] |
||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |