|
|
||||||||
Educational Methodologies |
Key words: problem-based learning, tutor, tutorial
Submitted for publication 11/15/06; accepted 02/08/07
| Abstract |
|---|
|
|
|---|
The experience of training faculty in the roles and responsibilities associated with problem-based learning has been more thoroughly explored in medical than in dental education. Reports from medical schools bring up a number of general issues to be considered when undertaking the creation of a faculty development program, such as sequencing, use of methodologies consistent with adult learning principles, institutional support, and program evaluation.25 Aside from these general considerations, there are additional specific issues that arise when considering a change in instructional approach to PBL. First, the transition from the traditional lecturer role to that of facilitator of learning is difficult for many faculty members.3,5 At the same time, the success of this transition may be critical to the success of the PBL pedagogy since there are important links between facilitator behavior and small group function.6,7
For these reasons, the developers of the faculty development program at USCSD decided that training in facilitation skills should be a major component in the design of the PBL faculty development program. It was felt, however, that faculty first needed to be equipped with a basic understanding of the PBL process. Accordingly, the Facilitation of Learning workshop was placed second in the workshop sequence, immediately following the PBL Process workshop, which was devoted to providing faculty with an understanding of the rationale and procedures involved in the PBL small-group learning process.8 The workshop planners recommended that the Facilitation of Learning workshop follow the PBL Process workshop and be in close conjunction with observation of a real PBL case. In addition, it was recommended that workshop participants should facilitate a case relatively soon after completing the workshops, so that individuals could apply what they had learned. The planners were committed to utilizing methods for instruction and practice that were consistent with concepts of adult learning theory9 and to providing opportunities for faculty to discuss their concerns amongst their peers in a nonthreatening environment. Program evaluation was also considered important to determine the impact of the program on faculty perceptions, as well as to assess the need for modifications to the program based on input from the faculty.
| Methods and Materials |
|---|
|
|
|---|
The formulation of the Facilitation of Learning workshop was based on the underlying idea that, to be effective as a facilitator, one should understand the process of learning being utilized by the student, the behaviors exhibited by the student that are representative of this process, and the general role the facilitator should take in guiding this process. All of these contribute to the goal that the participants would leave with the basic skills and confidence necessary to facilitate small-group student learning. It was also anticipated that workshop participants would leave with a more solid concept of the differences between traditional large-group teaching and small-group facilitating. With these items in mind, objectives for the workshop participant were set and are detailed in Table 1
.
|
The Facilitation of Learning workshop was presented in a half-day format. The sequence of activities in the workshop is summarized in Table 2
, including details of time allotted, the principle parties involved, and objectives addressed by each activity. The objectives for the workshop, as well as the workshop format, were presented to the participants at the beginning of the workshop, and the introduction also included a short recap of the PBL process as well as presentation of key concepts. The subject-specific scenario, role-modeling, and role-play activities are described in further detail in the next section. Selected literature on facilitation issues was made available to the participants at the close of the workshop as were two handouts, one outlining specific responsibilities of the facilitator and the other, a "Facilitation Starter Kit" designed by the program developers and based on recommendations from The Skilled Facilitator.10
|
|
|
|
![]() |
A set of criteria for facilitator behavior (Figure 4
), derived from a number of sources,10,12 was provided to the participants to aid their understanding and evaluation of the facilitators actions. A worksheet accompanying the list of criteria was designed to assist the participants in applying the criteria to the behaviors exhibited by members of the mock group. After behaviors and interventions had occurred, the demonstration was stopped, and a discussion of the events was initiated among the moderator, participants, and mock group members. The goal of the discussion was for the participants to link student behavior to the resulting facilitator intervention, applying the facilitator behavioral criteria and noting the impact of the facilitator on student behavior.
|
Methods of Program Evaluation
The Facilitation of Learning workshop was evaluated by surveying the participants immediately following completion of the workshop. The first part of the survey assessed the level of participant agreement that the workshop objectives had been met, using a four-point Likert scale with these choices: Strongly Disagree, Disagree, Agree, Strongly Agree. Survey data were tabulated by first assigning numerical values to the scale (1, 2, 3, and 4, respectively). Mean values and standard deviations were calculated for each survey item.
Participants were also asked to identify effective and ineffective methods used for the workshops and were asked to make any suggestions they might have as to how to improve the workshops in the future. Similar responses were grouped where possible to identify common recurrent themes. As an additional immediate assessment, those involved with running each workshop session discussed their impressions of the event with respect to logistics, participant outcomes, and needed modifications at the end of each presentation of the workshop.
| Results |
|---|
|
|
|---|
This article reports the outcomes of the PBL core skills workshops offered over a two-year period from 2003 to 2004. During this time, the Facilitation of Learning workshop was presented a total of eleven times. The number of participants enrolled in any given presentation of the workshop varied between five and twenty-five. While the workshops were open to all faculty and to non-faculty individuals involved in case facilitation, only those faculty employed with at least a 40 percent commitment of time to USCSD were required to be certified in the PBL core skills. Of the 182 individuals identified as requiring certification, 119 attended the Facilitation of Learning workshop during the indicated two-year period. All participants were given the opportunity to complete the exit survey regardless of faculty status or requirement that they be certified.
A total of 125 surveys were distributed to participants in the Facilitation of Learning workshop. Of these, eighty-nine were returned with data for analysis of responses regarding the achievement of workshop learning objectives, representing a 71.2 percent return rate for this portion of the survey. The analysis is summarized in Table 3
. In general, the respondents expressed the opinion that the workshop was successful as measured by their responses to the first part of the survey, in which they were asked whether they agreed or disagreed that the workshop had met its three stated objectives. The mean levels of participant agreement that the Facilitation of Learning workshop objectives had been met were 3.32 (SD ±0.63) for objective one, 3.34 (SD ±0.62) for objective two, and 3.36 (SD ±0.63) for objective three, where the numerical value of 3 was assigned to the "Agree" response choice and the value of 4 was assigned to "Strongly Agree."
|
The overall tone of the written comments was favorable in that there were more responses for the "most effective" question (89) than for the "least effective" question (55). Of the fifty-five respondents who made "less effective" entries, twenty-one wrote either "none" or "could not think of any" in response to the query, so that the number of actual negative assessments of the workshop obtained in response to this question was only thirty-four. In addition, there were twenty-two unsolicited expressions of either thanks or general praise for the workshops or the workshop leaders. Although they were not asked specifically about learning, thirteen of the participants volunteered the perception that they had learned something, either general or specific, and the area of perceived learning, when specified, was in all but one case related to one of the workshop objectives. These observations are broadly consistent with the view of the majority of participants that the workshop objectives had been met.
Comments Regarding Effectiveness of Workshop Exercises
The participants comments are summarized in Table 4
(most effective elements) and Table 5
(less effective elements). By far the greatest numbers of comments on effective elements of the workshop were related to the activities involving the mock group. In all, forty-two of the comments for the "most effective" question referred to some aspect of the mock group activities. Of those who wrote more extensive comments, many seemed simply to appreciate the opportunity to "see how it is done," as, for example, the participant who wrote that the most effective element was "being able to observe an experienced facilitator and group in action." A smaller number expressed an appreciation more specifically for what the activity was designed to show, as in the case of one participant who indicated that the effectiveness resulted from "actual observation of behaviors and how the facilitator managed the group." It was difficult to separate the role-modeling activity from the role-playing activity when analyzing the participants comments. Some mentioned role-playing specifically, but it was not always clear that the respondent understood the distinction between role-modeling and role-playing. In fact, these two activities were closely linked to each other in the workshop design, and it is not surprising that the participants might have seen them as a single activity.
|
|
A number of participant comments were related to what might be termed participant factors, outside of the control of the workshop organizers. Some of these were of interest in that they described participant input as an effective element and even commented that they had learned from fellow faculty or that "having people who had facilitated before coming to the workshop" was part of what made the experience effective. These comments may be viewed as further evidence that the workshop participants were actively interested in having additional input or insight into how PBL facilitation is done. The same may be said of the responses to the use of experienced PBL students as the "students" in the mock group in some of the workshops. A surprising number of participants mentioned that they found it helpful to hear the PBL students comments on their experiences in group or on what makes a good facilitator.
The mock group activities were also identified as a least effective element by some participants. However, none of the eighteen negative comments concerning the mock group was worded in such a way as to suggest that the mock group activity was viewed as ineffective. Rather, the comments referred to specific aspects of the activity, such as the length of the sessions, which did run over time on a number of occasions. Interestingly, a number of comments indicated that the respondents wanted more from these activities; that there was not enough attendee participation (role-playing) in the mock group sessions, or that the respondents thought too little of the case process was demonstrated. For example, one attendee wrote: "I think the group process should not have been interruptedI think it caused the group not to act all the way through a normal group process."
Items that could be termed participant factors also appeared in the ineffective comments. In contrast to those who appreciated having experienced facilitators among the participants, there were a few who felt that widely divergent levels of experience among the participants was a problem; that the presence of some who were "accomplished facilitators" inhibited the inexperienced participants from asking questions; or that some faculty participants dominated and were opinionated. The divergence of perspectives on the presence of participants with facilitation experience may reflect the individual makeup of the audiences at the different workshops.
Participant Suggestions
Fifty-four of the participants made responses in the space designated for suggestions, although fifteen of these responses were indications that the individual had no suggestion to offer, either because "everything was good" or because they felt they did not know enough to make useful suggestions. One participant wrote: "Maybe after I facilitate a group, I will have input in this category." Since the category of "suggestions for workshop improvement" was broad and open-ended, the observations here are, as one would expect, somewhat anecdotal. Nevertheless, it is worth noting that the general theme of participants wanting more experiences with observing or participating in small-group learning was evident in suggestions proposed by participants.
These suggestions included changes to the workshop itself and general improvements to facilitator training. Among examples of the former, a number of attendees suggested that videotaping real or mock sessions could be useful to make the mock sessions show what they were intended to show more efficiently or to provide more examples of good and bad facilitator behavior. One wrote, "Maybe there should have been film of other facilitators in action," and others suggested that it would be useful to have more feedback from experienced facilitators without referring to the use of film or videotape. As another example, several respondents offered suggestions to broaden the scope of coverage of the workshop to cover what is encountered in everyday facilitation, rather than just specific problem behaviors, or to cover how the facilitator deals with the full PBL process. (Because of the mini-case used, the mock group activity was confined to only the first iteration of the PBL cycle and in some cases did not even progress to the students selection of Learning Needs.) One attendee simply wrote that the workshop organizers should "have additional cases to present for more experience in handling problem situations." Finally, two additional suggestions related directly to increasing the participant involvement in the role-playing activity ("you need more participation by attendees"). Frequently, there was only time for one participant to take the facilitator role in the mock group, and some participants felt that the workshop could be improved by allowing more faculty to have this experience.
Participants suggestions for broader facilitator training experiences beyond the Facilitation of Learning workshop included such things as more observation of cases or being mentored on a case. One wrote: I "recommend that each participant sit in as [an] observer before taking on this as [an] independent solo facilitator." The original plan for this faculty development program included post-workshop observation, but this activity had not been implemented at the time of this study. One participant, writing in the "less effective elements" box, tellingly observed that "this process takes practiceanother case with practice facilitation would be helpful." On a related note, there were several suggestions for various kinds of ongoing follow-up to the workshop. The institution of regular short workshops was one of these. Another respondent wrote: "I think it would be a good idea if I, as a facilitator, have somebody I can go [to] and ask my questions that come out later!" Another suggested that the school should "develop a PBL facilitation website bulletin board or email to discuss difficult PBL dynamics."
| Discussion |
|---|
|
|
|---|
In evaluating the faculty development program instituted at USCSD, we found that participant responses to the Facilitation of Learning workshop were generally quite positive. Indeed, many participants freely expressed their appreciation of the learning opportunity afforded by the workshop activities. We further observed, however, that the workshops greatest perceived strength lay in the fact that it included an activity using a mock PBL group, thus providing participants an opportunity to observe, and in some cases participate in, aspects of the role of the PBL facilitator in action. In addition, a number of respondents voiced concerns that indicated they wanted more practice, more opportunities to observe experienced facilitators, or more instruction in aspects of the facilitators role that were not specifically included in the workshop design. One interpretation of these observations is that, faced with the major change in their role that PBL requires, most faculty were willing to learn, but felt insecure in their ability to fulfill the new role. They were glad to be offered helpful instruction, but they wanted more opportunities to observe or to practice facilitation and more demonstrations of additional functions they would be expected to perform. This is consistent with the view, also expressed by others, that acquiring confidence in ones ability to facilitate small-group learning requires extensive experience and practice over time.13
Achievement of Workshop Objectives
Based on the responses of the participants who filled out the survey, the workshop was broadly successful in achieving the stated objectives to increase understanding of the skills necessary to facilitate student groups, to increase the ability to identify student behaviors requiring facilitator intervention, and to increase understanding of the roles and responsibilities of the facilitator. On the face of it, faculty members perceptions that they have achieved a better theoretical understanding of a subject seem likely to be more accurate than a perception that one will be able to apply ones new understanding to a practical situation. Participant responses to a training program at the University of Natal support this idea as reported by Ohmesdahl and Manning.25 In that study, fifteen participants in a facilitator training program were surveyed immediately following a workshop and again after they had facilitated their first PBL module. The participants views remained fairly stable regarding agreement with statements on the effectiveness of small-group learning in achieving curricular goals and on the importance of group atmosphere to group goal achievement. A substantial number of the respondents indicated, however, that in practice they found themselves wanting more control over the students learning. They also reported having more difficulty avoiding providing information to students in the group than they had predicted immediately after completing the workshop.25 These points are included in ongoing planning to continue to improve the faculty as small-group facilitators.
Effectiveness of the Workshop Within the Program Sequence
The placement of the Facilitation of Learning workshop in the sequence following the PBL Process workshop proved effective. The intended order of the workshops was not rigidly enforced, but rather was presented to faculty as a recommendation. It was noted that some faculty members deviated from the recommended workshop sequence, probably for reasons of scheduling conflicts, and this may account for the exit survey comments of the few individuals who indicated that the workshops should be taken in order, that they would have liked to have seen more of the PBL process demonstrated in the facilitation workshop, or that the presence of participants who lacked an understanding of the PBL process was detrimental to the workshop experience.
There were a number of faculty members who had moderate facilitation experience prior to the development of the PBL Core Skills workshops. These individuals were waived from taking the PBL Process workshop but were still asked to attend the Facilitation of Learning workshop. This was done because it was felt that even experienced facilitators would likely benefit from the workshop content. In support of this judgment, only three survey respondents complained that the content was "redundant" or that there was "not much new" for them in the Facilitation of Learning workshop.
Effectiveness of Application of Adult Learning Principles
Consistent with the principle that adults like to be actively involved in their learning rather than mere passive recipients, there was evidence that the participants valued the opportunities to participate actively in the workshop and that they valued the participation of the other workshop attendees as well. This was revealed in part by the number of comments, already mentioned, that pointed to "discussions" of one kind or another as effective elements. In addition, there were six comments in the "effective" box that cited input from the workshop audience specifically.
The striking popularity of the activities involving the mock group may, in part, reflect the adult preference for lifelike learning presentations. Four comments made direct reference to the mock sessions "realism." For many of the other respondents, whose comments were grouped under "the mock group (role-modeling and/or role-play)," it was apparent that having the opportunity to "see how its done" by an experienced facilitator was valued as a learning experience. In this case, it may simply be that facilitation is a complex skill that is better learned through observation than through didactic instruction.
Lessons for the Future
A number of things were learned from the survey results and from the experience of running the workshop program that may be used to modify or extend the program in the future. The workshop could be strengthened by expanding the popular mock group activity, though not at the expense of other aspects of the workshop program, especially activities that provided the participants an opportunity to ask questions and freely express their concerns. Modifications to the mock group activity that we are considering include 1) focusing less on specific problem behaviors in the mock group sessions by including more on the basics of working with a group, 2) covering more parts of the case, and 3) giving all participants the opportunity to practice being the facilitator.
We are looking into the possibility of preparing videotaped demonstration sessions to provide additional opportunities to "see how its done." Such tapes could be made available online for faculty to view at their own convenience. In addition, it is clear that our intention to have faculty observe a case being facilitated by an experienced faculty member as part of their training is a plan that should be further implemented, as it would provide more extensive demonstration, including the opportunity to see all parts of the case demonstrated. Having the novice facilitators first case observed by an experienced peer who can provide ongoing evaluation and mentoring is likely to enhance the overall training experience. Finally, in recognizing the fact that facilitation of learning is a complex and subtle, as well as an unfamiliar, skill for most faculty, the need for an ongoing program of observation of facilitator performance, combined with peer mentoring and follow-up learning opportunities, cannot be overemphasized.
| Conclusions |
|---|
|
|
|---|
| Acknowledgments |
|---|
| Footnotes |
|---|
| REFERENCES |
|---|
|
|
|---|
This article has been cited by other articles:
![]() |
D. O. B. Lam Impact of Problem-Based Learning on Social Work Students: Growth and Limits Br. J. Soc. Work, May 7, 2008; (2008) bcn073v1. [Abstract] [Full Text] [PDF] |
||||
![]() |
K. R. Dalrymple, S. Wong, A. Rosenblum, C. Wuenschell, M. Paine, and C. F. Shuler PBL Core Skills Faculty Development Workshop 3: Understanding PBL Process Assessment and Feedback via Scenario-Based Discussions, Observation, and Role-Play J Dent Educ., December 1, 2007; 71(12): 1561 - 1573. [Abstract] [Full Text] [PDF] |
||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |