| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
Chapter 4 |
Key words: longitudinal evaluation, multimethod evaluation, multiple case studies, evaluation design, health professions education reform, community-based dental education
Previous chapters of this report described the dental care access crisis and workforce diversity concerns that stimulated the Robert Wood Johnson Foundation (RWJF) Human Capital initiative to sponsor the Pipeline, Profession, and Practice: Community-Based Dental Education program.1,2 The Pipeline program aims to build a strong, capable, and diverse workforce within the field of dentistry, equipped to address the critical shortage of oral health care services for underserved and disadvantaged populations. In 2001, RWJF provided funding to eleven dental schools to develop and execute a Pipeline program; a National Program Office (NPO) to provide leadership and technical support to the schools; and a National Evaluation Team (NET) to comprehensively evaluate the initiative. In 2003, The California Endowment (TCE) provided funding to establish Pipeline programs at four additional dental schools in California and to create a statewide recruitment and health policy initiative. Figure 4.1
shows the geographic location of the fifteen Pipeline dental schools.
|
This chapter describes the process for conducting a large-scale longitudinal evaluation and summarizes the methods used in data collection and analysis. The evaluation process was initiated by successfully competing for the grant to evaluate the Pipeline program. The broad policy questions related to workforce diversity and oral health care access were translated into concrete and measurable evaluation questions. In turn, these questions led to the design of the evaluation framework or logic model. The framework was used as a guide for specifying and operationalizing variables for assessing the effects of context, structure, and process on the intermediate and longer term program outcomes. We identified relevant stakeholders and used existing data sources or created new data sources if none previously existed. Various evaluation designs were used in the study, depending on the evaluation question and data sources used to address the questions. Our abundance of data sources led to many opportunities and challenges for data triangulation. This study is the first longitudinal evaluation to systematically assess changes in U.S. dental schools to improve access to oral health care in underserved populations.
 |
Ingredients for a National Evaluation |
|---|
 TopAuthor information Ingredients for a national... Evaluation process and phasesSummary and ConclusionsReferences |
|---|
The NET was comprised of an interdisciplinary group of investigators from dentistry, medical sociology, health services policy and management, health program evaluation, and biostatistics. In the formative stages, we consulted with a qualitative researcher and qualitative software vendors. Additionally, the ever-present NPO consulted frequently with the NET and Pipeline school investigators to provide technical assistance regarding program design and execution and database development, such as a clinical information system to assess progress at the community-based clinical rotation sites.
Finally, always essential to the NET operations were the competent project manager, research associates, programmer-analyst, and research assistants. When we were not exploring data collection methods most acceptable to the dental schools or collecting and analyzing data, our project manager and staff spent their time planning multiple site visits to the fifteen Pipeline dental schools. A logistical challenge on the best of days, we were extremely fortunate to hold onto our key project management and research staff over the full course of the evaluation. Performance matters, and staffing is a critical resource that should never be undervalued or taken for granted in achieving an effective and well-managed evaluation.
A final critical ingredient for the national evaluation was communication, communication, and more communication. Effective and ongoing internal and external communication is essential when tackling a data collection and analysis effort of this scope. Our project manager spent approximately 80 percent of his time interacting with the multitude of stakeholders engaged in the Pipeline program: the individuals, groups, or organizations that have a vested interest in how well the health program functions.4 Equally challenging was keeping the investigator team aligned to complete the comprehensive evaluation using scientifically valid and reliable methods of data collection. Biweekly NET team meetings were interspersed with frequent workgroup meetings organized around the major evaluation questions (recruitment, curriculum, etc.). Meetings were organized by project leadership and staff to support the investigators and maintain project alignment. The primary role of the workgroups was to develop and execute data collection and analysis plans.
|
Evaluation Process and Phases |
|---|
|
TopAuthor informationIngredients for a national...Evaluation process and phasesSummary and ConclusionsReferences |
|---|
Most professional evaluators prefer to be involved in a given project early and, often, long before the program is implemented. The NET successfully competed nationally for the Pipeline program evaluation grant and spent more than a year in planning. If evaluation is not considered early in the program planning process, remedial work is often required to design and implement a meaningful evaluation of the outcomes and impact. Retrospective evaluations are often less useful because opportunities for primary data collection and selection of an appropriate evaluation design are limited once the program has been implemented. Prospective evaluations such as that for the Pipeline program, on the other hand, can be much more useful and comprehensive because early entry into the conceptualization and design phases provides a basis for a more comprehensive and systematic evaluation of outcomes.
Our comprehensive evaluation process essentially consists of six phases: conceptualization, design, implementation, impact assessment, data triangulation, and dissemination.
Phase I. Conceptualization
Conceptualization generally includes a series of activities starting with understanding the scope and magnitude of the health problem and the rationale and evidence for developing a program to address it. The two major activities that occur in Phase I are 1) translating policy issues to evaluation questions5 and 2) developing an evaluation framework or logic model.
Translating Policy Issues to Evaluation Questions.
For the Pipeline evaluation, the major policy issue is the oral health care access crisis and the role workforce diversity may play in helping to ameliorate the crisis. The Pipeline program was funded to transform the culture of dental schools and the training of dentists to improve access in underserved and disadvantaged communities. From this policy analysis, the NPO designed the program and defined its major programmatic objectives. At the planning stage for the longitudinal evaluation, the NET worked with the foundation officers to translate the policy issues and Pipeline program objectives into concrete evaluation questions.5
Policy Issue: The nation’s dental care safety net is limited, and the access problems will become more acute in the next decade as the relative supply of dentists declines.
Policy Question: How can universities and dental schools play a more active role in improving access to dental care for under-served and disadvantaged populations?
Select Evaluation Questions:
Evaluation Framework to Guide Data Collection and Analysis.
Chapter 3 provided a detailed explanation of the evaluation framework and comprehensive literature review.2 This chapter describes the process and utility of designing the framework as part of the comprehensive evaluation process. After agreement is reached vis-à-vis the evaluation questions, the next step involves concept mapping, which ultimately yields an evaluation framework or "logic model."6 Concept mapping is used to develop the components of the evaluation framework, articulate a theory of change, and surface the key study variables for the evaluation.7 Early in the conceptualization phase, the evaluation framework is used to better understand the underlying implicit and explicit rationales and elements of the program and evaluation efforts, as well as their relationships and causalities. The evaluation framework is used to guide data collection and analysis. Since it guides the planning of the evaluation effort, building the evaluation framework is one of the core competencies required for an effective evaluation process. The Pipeline program evaluation framework reflects the six major evaluation questions addressed in this longitudinal evaluation.
The framework, shown in Figure 4.2, was adapted from our previous conceptual and analytical work on medical care access8–10 and the structures, processes, and outcomes of educational programs in achieving both short-term educational outcomes and longer term benefits to organizations, delivery systems, and society.11,12 Our previous research combined with a comprehensive literature review and the six evaluation questions were used to create the conceptual framework.
|
The evaluation framework maps relations and causalities between the elements and change targets, and illustrates how the program is directly linked to and impacts on the intermediate and longer-term outcomes. It also justifies needed resources and support for the program and offers opportunities to apply evaluation research to monitor program design, implementation, and outcomes. The evaluation framework uses a single graphic representation to rapidly communicate concepts that might otherwise require substantial text and explanation to convey. The framework is a critical communication vehicle used by the NET leadership to keep the interdisciplinary team aligned.
Phase II. Design
Design requires technical knowledge related to 1) understanding how to craft an evaluation plan that is parallel to the intervention plan, 2) collecting valid and reliable data, and 3) selecting an evaluation design that is feasible and practical to implement. Keeping in mind that comprehensive evaluations are best planned concurrently with program planning, Pipeline program objectives were included as part of the evaluation request for proposal (RFP). The NET planned the longitudinal evaluation at the same time the schools were designing their Pipeline programs—a parallel schedule that is optimal.
Once the NET agreed on an evaluation framework to guide data collection and analysis, our next task was to move from concept to measurement. The most efficient approach is to identify existing data sources already tested for validity and reliability. Primary data collection is much more costly in terms of personnel and other resource costs since it requires Institutional Review Board (IRB) approval, pilot-testing, revising, administering, multiple call-backs to increase response rates, and processing and cleaning data. In the Pipeline program evaluation, it was necessary to develop new data sources where none previously existed. Table 4.1 presents a summary of the data sources, including unit of analysis (e.g., individual student or faculty member, school, county, state), purpose for the data source, data collection methods, timing for data collection, evaluation design, and data analysis.
|
shows the qualitative site visit interview data were collected from multiple stakeholder groups in three waves at the RWJF-funded schools and in two waves at TCE-funded schools. Two-day site visits were conducted by one to three of the NET co-investigators. Thirteen to eighteen interviews generally lasting one hour each were conducted at each site visit. About half of the interviews were with individual stakeholders, and half were group interviews. The group interviews were conducted with two to four faculty members, administrators, or community representatives and three to nine first- or fourth-year students. The interviews were taped, transcribed, and analyzed using N6 NUD*IST Software for Qualitative Data Analysis. We collected the site visit interview data for several purposes: to describe Pipeline program components including baseline status, program structures, and implementation processes; to validate and clarify information gathered from other data sources; to collect information not available from other data sources; and to identify evidence-based best practices in Pipeline schools. Interview data were collected using a uniform bank of questions to address the evaluation questions; for example, data were used to assess the community-based dental education curriculum at baseline, content of existing courses, plans for changing the curriculum, and barriers and facilitating factors to implementing curricular changes.
Implementation reports were provided annually by all grantees (see Table 4.1). These reports included administrative data on the structure, processes, and outcomes of the Pipeline program components. For example, schools provided information on the number and type of extramural clinical rotation sites (structure), strategies for developing community partnerships (processes), and the number of hours students spent providing care in the community (outcome). Annual financial reports, also collected each year, show changes in revenues and expenses related to moving clinical education to community settings. These data have obvious implications for sustaining community-based dental education programs.
We were fortunate to have access to two existing data sources collected annually from all accredited U.S. dental schools: the American Dental Education Association (ADEA) survey of dental school seniors, and the American Dental Association (ADA) survey of predoctoral dental education. The NET collaborated with ADEA to update and revise the existing senior survey to measure changes in community-based dental education and clinical practice associated with the Pipeline program. The senior survey collects information about the financing of dental education, graduating indebtedness, practice and postdoctoral education plans following graduation, decision factors that influenced postgraduation plans, and impressions of the adequacy of time directed to various areas of predoctoral education. Each school uses its own survey distribution and collection system to conduct the survey. Fifty-two accredited dental schools returned surveys in 2003, resulting in an overall student response rate of 85 percent. In 2007, the response rates for Boston University and the University of Maryland/Baltimore College of Dental Surgery were so low that the ADEA data for 2006 from these two schools were used instead for the evaluation analysis, resulting in an overall response rate for the fifty-two schools in 2007 of 86 percent. The annual ADA survey was most useful for assessing URM student recruitment in both Pipeline and non-Pipeline dental schools. This survey collects data each year from each dental school on enrollment numbers and the ethnic composition of the student body.
The NET created a new data source to survey the faculty at Pipeline dental schools. The faculty survey was conducted twice during the Pipeline years, in 2004 and 2006. This survey contained questions to address all of the evaluation questions. As well, some of the questions reflected items in the ADEA survey of dental school seniors, allowing the NET to assess the gaps between student and faculty perceptions of the Pipeline program. The survey was administered to all dental schools and faculty in these categories: 1) appointment equal to or greater than 40 percent time; 2) basic science course chairs; and 3) community dentists who supervised one or more students in the past two years. The survey for 2004 was administered in the fall and winter of 2004–05, and the survey for 2006 was administered in the fall and winter of 2006–07. Faculty at both dental schools and extramural sites were sampled at fourteen Pipeline schools. Schools had the option of using online or paper versions of the survey. For the 2004 survey, 1,053 surveys were received for a response rate of 58 percent. For the 2006 survey, 1,027 surveys were returned for a response rate of 60 percent.
Two major data sources were created to describe and monitor curricular revisions. Syllabi were collected during the site visit years to assess community-based dental education (CBDE) curricular revisions and to identify innovative courses and teaching methods. In the final program year, the NET developed a curriculum checklist that each Pipeline school used to summarize its CBDE curricular changes over the course of the Pipeline program.
The NET and NPO provided technical assistance to the schools for developing a clinical information system (CIS) containing a uniform set of data elements collected at the extramural rotation sites. These data report the type and volume of services provided by students in the community sites. Other secondary data sources were used to construct contextual variables. In this evaluation, contextual variables represent the policy, delivery system, university and school, and population characteristics influencing dental programs, educational outcomes, and dental care access.9,11-15
Phase III. Implementation
Implementation is primarily concerned with the extent to which the intervention is implemented as planned.16,17 The NPO required each Pipeline program to create an implementation plan, timelines, and a sequencing of key activities and/or events for each Pipeline objective. The NPO was also responsible for monitoring progress to ensure the proper and timely execution of the intervention; adherence to predetermined time schedules, budget, and use of resources; and documentation of instances when the implementation plan changed, exceeded, or failed to meet expectations.
The primary data source for monitoring implementation was the annual implementation report on program plans and progress, collected by the NPO and shared with the NET. Since resources are required to collect, process, and monitor data, the NET specified the critical path for implementation and then selected key process variables for each of the program components to emphasize in the evaluation. Table 4.2 shows key process measures, data sources, and other study variables emphasized in the longitudinal evaluation.
|
Technical knowledge and skills are required to design an impact assessment. As shown in Figure 4.3, evaluation designs are generally described as pre-experimental, quasi-experimental, or experimental. Pre-experimental designs are the weakest of the three and are often used to test pilot projects in the formative stage of development when the evidence base is limited. Pre-experimental designs often use a single case study, a one group pretest-posttest design, or a case control study.5,18,19
|
), where "0" indicates an observation of the outcome or dependent variable and "X" indicates the program or intervention using Campbell and Stanley notation.18
|
). Four tests are relevant for judging the quality of the case study results: construct validity, reliability, internal validity, and external validity.19 Construct validity requires establishing correct operational measures for the concepts studied. To establish construct validity we used multiple sources of evidence and obtained reports from multiple stakeholder groups; established a chain of evidence over the course of the Pipeline program years; and had key informants, specifically the principal investigator from each school, review the case study report for accuracy and to correct any errors in fact.
Reliability was strengthened by using a uniform case study protocol and developing a case study database. In the case study design, internal validity was established through explanation building over time and the time series design drawing on multiple data sources from 2002 (the school’s Pipeline proposal) through 2007. External validity was established through replication by analyzing data from multiple case studies, shown graphically in Figure 4.5.
|
Quasi-experimental designs are used when experimental designs are not feasible or when they are difficult to implement. In contrast to true experiments, which generally use random assignment to create comparison groups, quasi-experiments depend on non-equivalent groups that occur naturally in the social environment.18 A frequently applied quasi-experimental design—the pretest-posttest non-equivalent comparison group design—was used in the NET evaluation. Primarily drawing on the ADEA, ADA, and contextual variable data sources, we conducted impact assessments using multivariable, multilevel linear statistical models. The analyses presented in Chapters 6.0 through 6.635-41 of this report compare changes in Pipeline program schools versus non-Pipeline schools from baseline (2002–03) to the culmination of the program (2006–07) using the design shown in Figure 4.6. The dotted line indicates the groups are non-comparable at baseline. In general, the California Pipeline schools (n=5) were implemented about six months following the other Pipeline schools (n=10), and the non-Pipeline schools (n=38) are the accredited dental schools not funded for a Pipeline program.
|
As indicated in our evaluation framework, the key variables important for the longitudinal evaluation can be specified and operationalized once the evaluation design has been selected. As mentioned previously, the evaluation framework was used to guide selection of the major study variables emphasized in the longitudinal evaluation (Figure 4.2). (Appendix 1 of this report contains a complete listing and operational definitions of all variables analyzed in the NET longitudinal evaluation.42)
Table 4.2 outlines the six areas reflecting the evaluation questions addressed in the study. NET workgroups assigned to each evaluation question identified the major study variables associated with each question including context, structure, process, and outcomes. The first three areas reflect the Pipeline program (recruitment, curricular revisions, and extramural clinical rotations), and the latter three reflect longer term outcomes associated with the program (practice plans, sustainability, and policy reform). The evaluation framework is organized by context, structure, process, and outcome variables emphasized in the study within each conceptual domain.
Typically, there are three steps involved in evaluating the impact of a program. First, the impact is assessed using methods and analyses appropriate for the evaluation design, level of measurement (individual or school), and timing of the outcome measures. Second, both quantitative and qualitative data are used for analyses to explain the efficacy and results of the health program. Third, when possible, cost and resource analyses are conducted to assess the efficiency of the program and its components.
Phase V. Data Triangulation
In this longitudinal evaluation, we created enormous opportunities for data triangulation, drawing on ten data sources collected from the multiple stake-holder groups discussed above. Data triangulation involves using multiple methods of data collection and analysis to strengthen the validity of findings, if results produced by different methods are congruent (Figure 4.7). Using multiple methods is a means of offsetting different kinds of bias and measurement error.
|
outlines key context, structure, process, and outcome variables considered for evaluating the effects of the Pipeline program on changing the practice plans of graduating seniors. This analysis used multiple evaluation designs, multiple methods of measurement, and data collection and analysis in evaluating the effects of the program.5,17 The major outcome variables were 1) percentages of underserved minority patients dental school seniors plan to care for upon graduation, and 2) percentages of seniors planning to practice in community clinics or government settings, before and after the Pipeline program was implemented. To address the evaluation questions, we triangulated qualitative site visit interview data from multiple stakeholder groups to learn about the financial barriers to providing care to low-income uninsured patients, including substantial educational debt upon graduation and low reimbursement for public insurance programs. Additionally, the faculty survey data were used to assess not only barriers to providing care, but also information sources for learning about opportunities to care for underserved patients (community rotation sites, dental safety net organizations, didactic courses, etc.). As stated earlier, data collected in the annual ADEA survey of dental school seniors were used to examine changes and trends in community-based dental education and practice plans of successive cohorts of dental school seniors. Our impact assessment applied a quasi-experimental design with two intervention groups (National Pipeline schools and California Pipeline schools) and a non-equivalent comparison group (non-Pipeline schools). Multivariable, multilevel linear statistical models were used to determine the effects of the Pipeline program, after adjusting for significant student, educational program, and contextual determinants of practice plans upon graduation. The school-level analyses compared the adjusted outcome measures in the Pipeline and non-Pipeline schools pre- and post-Pipeline implementation. The student-level analyses compared the adjusted outcome measures in the Pipeline and non-Pipeline schools for the cohorts that preceded and followed Pipeline implementation, while also adjusting for possible intra-class correlation in the dental schools.
Thus, triangulation involving multiple data sources, data collection, and analysis methods from multiple stakeholders and multiple evaluation designs can be used to strengthen the results of the evaluation. However, there is always a chance triangulation may yield incongruent results and seemingly conflicting findings; then, it is up to the evaluator to try to explain these results by closely examining the validity and reliability of the data and other measurement issues including random error, systematic error, and measurement error, which may contribute to disparate evaluation findings. Chapter 6.4 of this report investigates the facilitating factors and barriers to providing care to underserved patients upon graduation.40 We used site visit interviews and faculty survey data to address this question. From the site visit interviews, we found discordant views among the senior dental school seniors, who were far less convinced than administrators and faculty members about the extent to which schools communicated about opportunities to practice in underserved areas. In this example, differences in stakeholder perceptions accounted for inconsistent results.
Phase VI. Dissemination
This chapter has described the evaluation process and phases of a comprehensive evaluation, from conceptualization to design to implementation to impact assessment to data triangulation. In this final phase, the findings—including implications for improving state and federal policy, health care delivery system financing and workforce, university policies, educational programs and outcomes, and population access—are reported to the sponsoring foundations and other relevant stakeholders. Since the first Pipeline program is quickly being followed by Pipeline II, the NET had a powerful incentive to share the evidence in a timely fashion, so the evaluation results can be used to immediately and directly inform the next phases of program design and execution.
Over the years, our primary dissemination channels have been two annual conferences hosted each year by the National Program Office, feedback reports to the schools after each site visit, ongoing reports to the foundation officers, numerous journal articles along the way, and this report published as a special supplement to the Journal of Dental Education, which will be widely disseminated by means of a national communications campaign. As well, the National Program Office spearheaded a communications strategy throughout the Pipeline years to stimulate fieldwide interest in the program, which had the potential for creating a positive secular trend in the dental education field.
|
Summary and Conclusions |
|---|
|
TopAuthor informationIngredients for a national...Evaluation process and phasesSummary and ConclusionsReferences |
|---|
|
Author Information |
|---|
|
TopAuthor informationIngredients for a national...Evaluation process and phasesSummary and ConclusionsReferences |
|---|
|
REFERENCES |
|---|
|
TopAuthor informationIngredients for a national...Evaluation process and phasesSummary and ConclusionsReferences |
|---|
This article has been cited by other articles:
|
|
 |
|
  R. M. Andersen and P. L. Davidson Introduction to the Evaluating the Dental Pipeline Program Report J Dent Educ., February 1, 2009; 73(2_suppl): S10 - 14. [Full Text] [PDF]
|
|
|
|
|
|
J. J. Crall, E. R. Hewlett, J.-A. Friedman, A. K. Mascarenhas, S. Freilich, M. Henshaw, J. A. Jones, M. Mann, and S. Frankl The Pipeline Program at Boston University Goldman School of Dental Medicine J Dent Educ., February 1, 2009; 73(2_suppl): S58 - 69. [Full Text] [PDF]
|
|
|
|
|
|
R. M. Andersen, K. A. Atchison, E. R. Hewlett, and D. Grant-Mills The Pipeline Program at Howard University College of Dentistry J Dent Educ., February 1, 2009; 73(2_suppl): S70 - 82. [Full Text] [PDF]
|
|
|
|
|
|
J.-A. Friedman, A. Thind, P. L. Davidson, and C. Farmer-Dixon The Pipeline Program at Meharry Medical College School of Dentistry J Dent Educ., February 1, 2009; 73(2_suppl): S83 - 95. [Full Text] [PDF]
|
|
|
|
|
|
A. Thind, E. R. Hewlett, R. M. Andersen, and C. Y. Bean The Pipeline Program at The Ohio State University College of Dentistry: Oral Health Improvement through Outreach (OHIO) Project J Dent Educ., February 1, 2009; 73(2_suppl): S96 - 107. [Full Text] [PDF]
|
|
|
|
|
|
P. L. Davidson, A. Thind, J.-A. Friedman, D. C. Carreon, and C. Hodge The Pipeline Program at the University of Connecticut School of Dental Medicine J Dent Educ., February 1, 2009; 73(2_suppl): S108 - 119. [Full Text] [PDF]
|
|
|
|
|
|
K. A. Atchison, E. R. Hewlett, J.-A. Friedman, C. A. Evans, A. J. Bolden, and C. Hryhorczuk The Pipeline Program at the University of Illinois at Chicago College of Dentistry J Dent Educ., February 1, 2009; 73(2_suppl): S120 - 134. [Full Text] [PDF]
|
|
|
|
|
|
E. R. Hewlett, R. M. Andersen, K. A. Atchison, and R. P. Strauss The Pipeline Program at the University of North Carolina at Chapel Hill School of Dentistry J Dent Educ., February 1, 2009; 73(2_suppl): S135 - 147. [Full Text] [PDF]
|
|
|
|
|
|
K. A. Atchison, J.-A. Friedman, J. R. Freed, and D. Jackson The Pipeline Program at the University of Washington School of Dentistry J Dent Educ., February 1, 2009; 73(2_suppl): S148 - 160. [Full Text] [PDF]
|
|
|
|
|
|
J.-A. Friedman, E. R. Hewlett, K. A. Atchison, and S. S. Price The Pipeline Program at West Virginia University School of Dentistry J Dent Educ., February 1, 2009; 73(2_suppl): S161 - 174. [Full Text] [PDF]
|
|
|
|
|
|
J. J. Crall, J.-A. Friedman, K. A. Atchison, and C. J. Goodacre The Pipeline Program at Loma Linda University School of Dentistry J Dent Educ., February 1, 2009; 73(2_suppl): S175 - 185. [Full Text] [PDF]
|
|
|
|
|
|
H. C. Gift, R. M. Andersen, P. L. Davidson, A. Thind, J. M. Yamamoto, C. A. Maida, and M. Marcus The Pipeline Program at the University of California, Los Angeles, School of Dentistry J Dent Educ., February 1, 2009; 73(2_suppl): S186 - 198. [Full Text] [PDF]
|
|
|
|
|
|
E. R. Hewlett, R. M. Andersen, K. A. Atchison, and W. Bird The Pipeline Program at the University of California, San Francisco, School of Dentistry J Dent Educ., February 1, 2009; 73(2_suppl): S199 - 210. [Full Text] [PDF]
|
|
|
|
|
|
A. Thind, R. M. Andersen, P. L. Davidson, and P. Glassman The Pipeline Program at the University of the Pacific Arthur A. Dugoni School of Dentistry J Dent Educ., February 1, 2009; 73(2_suppl): S211 - 221. [Full Text] [PDF]
|
|
|
|
|
|
P. L. Davidson, R. M. Andersen, A. Thind, R. Mulligan, and N. Nathason The Pipeline Program at the University of Southern California School of Dentistry J Dent Educ., February 1, 2009; 73(2_suppl): S222 - 235. [Full Text] [PDF]
|
|
|
|
|
|
J. J. Gutierrez, T. T. Nakazono, D. C. Carreon, and R. M. Andersen Introduction to the Cross-Site Comparisons and Multivariable Analyses of the Dental Pipeline Program J Dent Educ., February 1, 2009; 73(2_suppl): S236 - 237. [Full Text] [PDF]
|
|
|
|
|
|
R. M. Andersen, J.-A. Friedman, D. C. Carreon, J. Bai, T. T. Nakazono, A. Afifi, and J. J. Gutierrez Recruitment and Retention of Underrepresented Minority and Low-Income Dental Students: Effects of the Pipeline Program J Dent Educ., February 1, 2009; 73(2_suppl): S238 - 258. [Full Text] [PDF]
|
|
|
|
|
|
E. R. Hewlett, P. L. Davidson, T. T. Nakazono, D. C. Carreon, J. J. Gutierrez, and A. Afifi Revisions to Dental School Curricula: Effects of the Pipeline Program J Dent Educ., February 1, 2009; 73(2_suppl): S259 - 268. [Full Text] [PDF]
|
|
|
|
|
|
P. L. Davidson, T. T. Nakazono, D. C. Carreon, J. Bai, and A. Afifi Practice Plans of Dental School Graduating Seniors: Effects of the Pipeline Program J Dent Educ., February 1, 2009; 73(2_suppl): S283 - 296. [Full Text] [PDF]
|
|
|
|
|
|
J. J. Crall, P. L. Davidson, T. T. Nakazono, J. J. Gutierrez, J. Bai, and R. M. Andersen Involvement in Health Policy Regarding Oral Health and Dental Education: Effects of the Pipeline Program J Dent Educ., February 1, 2009; 73(2_suppl): S308 - 318. [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
