JDE
HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS
QUICK SEARCH:   [advanced]


     

J Dent Educ. 69(2): 285-295 2005
© 2005 American Dental Education Association
This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Holtzman, J. M.
Right arrow Articles by Sanders, R. M.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Holtzman, J. M.
Right arrow Articles by Sanders, R. M.

Educational Methodologies

Computerized Assessment of Dental Student Writing Skills

Joseph M. Holtzman, Ph.D.; Norbert Elliot, Ph.D.; Cheryl L. Biber, D.M.D., M.P.H.; R. Michael Sanders, D.M.D., Ed.M.

Key words: dental curriculum, computer-assisted instruction, communication, educational measurement

Submitted for publication 06/28/04; accepted 11/02/04


   Abstract
Top
 Abstract
The ability to write...
 The present study
 Materials and Methods
 Results
 Discussion
 Conclusions
 References
 
This study tested the feasibility of using computer-based tools for the assessment of written materials produced by dental students. Written assignments produced by three consecutive incoming dental school classes (240 students) were assessed, and the performance among and between classes was analyzed. Computerized assessment of documents produced by students in the context of their regular coursework proved to be an efficient and effective mechanism for assessing performance. Student performance, assessed as a byproduct of this research, was disappointing. The performance of all classes fell below the eleventh grade level, with some students producing written material at a level of sophistication generally expected from middle school children. Existing technology shows promise as a vehicle for enhancing the assessment of dental students’ written communication skills. The ease of use and minimal training necessary to apply this technology can help mitigate the time-intensive nature of writing assessment. If this assessment information is then used to enhance instruction—a process inherently available through software such as WebCT—the distance between assessment and instruction may be more readily bridged through an increase in the use of technology.

Historically, the ability to communicate effectively has been recognized as a hallmark for membership in the learned professions. During nineteenth-century America, the promotion of basic literacy—the ability to read newspapers and write letters—was associated with the beginning of industrialization itself. Today, the association between advanced literacy—the ability to read a variety of texts in a critical fashion and to write in a variety of modes for diverse audiences—and professional advancement continues.1–3 While this kind of advanced literacy was simply unavailable at the close of the nineteenth century (only 2 percent of the eighteen- to twenty-four-year-old population attended college in 1899–1900), such is not the case today.4 As noted recently in the National Commission on Writing in America’s School and Colleges, "When education was a private good, available to only a small elite in the United States, grammar, rhetoric, and logic were considered to be the foundation on which real learning and self-knowledge were built."5 However, as the commission continued, "Writing today is not a frill for the few, but an essential skill for the many." That "many" is predicted to be phenomenally large: 18.8 million students are predicted to have seats in our nation’s college classrooms by 2013.6


   The Ability to Write Effectively
Top
 Abstract
 The ability to write...
The present study
 Materials and Methods
 Results
 Discussion
 Conclusions
 References
 
Advanced written communication skills (the ability to write in a variety of modes to a variety of audiences) are essential in dental practice and dental education. Dentists and dental educators must competently produce a wide range of written communications including referral letters, communications with insurance companies, instructional procedures to laboratories, scholarly publications, and patient education materials. The American Dental Association has incorporated publication as an aim of professional responsibility, a practice that carries with it the demand for a high level of writing ability. In addition, the practice of writing carries psycho-dynamic properties; therefore, dental educators have advocated the incorporation of analytical writing tasks into the curriculum to provide opportunities for students to develop their reflective, critical thinking abilities.7

There have been few published studies of the quality of dentist-authored written communication; however, those that have appeared suggest that significant deficiencies exist in the quality of these materials.8,9 As recently noted by Alexander, appropriate written communications can be an important supplement to verbal instructions and can improve both patient satisfaction and compliance.10 It is Alexander’s observation that dentistry has lagged behind the other health professions in the systematic evaluation of written material provided to patients.

Yet, if such written materials are lacking in their design, it must also be recognized that dental education is but part of a complex system in which there is no guarantee that entering students possess the advanced writing skills necessary to communicate with a diverse patient population. Studies of the writing performance of high school students and college freshmen continue to suggest that the achievement of writing ability is a complex, difficult goal of education at all levels. A recent report on California’s colleges and universities found that less than 50 percent of freshmen could produce technically adequate papers and did not demonstrate the ability to analyze arguments or synthesize information.11 The commission stated, "What most students cannot do is write well. At least, they cannot write well enough to meet the demands they face in higher education and the emerging work environment. Basic writing itself is not the issue; the problem is that most students cannot write with the skill expected of them today." Further, citing the National Assessment of Education Progress, the commission report states that "most students have mastered writing basics, but few are able to create precise, engaging, coherent prose." To paraphrase a recent article in the Chronicle of Higher Education, Johnny may not be able to write, even if he went to Princeton.12

Nevertheless, dental professionals need advanced writing skills to function as competent health care providers. In the early 1980s, Bjork13 identified the need to incorporate training in scientific writing into the preparation of dentists and described the philosophy, structure, and content of a professional writing course for dentists; such courses do not appear to have become an integral part of the undergraduate dental curriculum. The reasons are evident: assessing writing, as well as teaching it, is complex, difficult work. The National Commission on Writing in America’s School and Colleges acknowledges such difficulties and suggests increased reliance on information technology to enhance students’ skills in written communication. The commission recommended that private and public leaders "work with educators to apply new technologies to the teaching, development, grading, and assessment of writing" and that "the nation should invest in research that explores the potential of new and emerging technologies to identify mistakes in grammar, encourage students to share their work, help assess writing samples, and incorporate software into measuring student-writing competence." If even the National Commission suggests that the time necessary to assess, correct, and give feedback to students posed significant barriers to the implementation of writing programs, then what is the profession of dentistry to do in order to assess and increase the writing abilities of its students?


   The Present Study
Top
 Abstract
 The ability to write...
 The present study
Materials and Methods
 Results
 Discussion
 Conclusions
 References
 
In this study, we investigated the feasibility of using the most widely available computer-based tools—those embedded in Microsoft Word—to assess dental students’ writing skills. We recognized the limits of the study at the onset. Our approach did not allow us to investigate aspects of writing in as robust a way as those afforded by computer-based systems such as Criterion (developed by the Educational Testing Service), iMOAT (developed by Massachusetts Institute of Technology and the Microsoft iCampus Alliance), or more advanced approaches recently described by Burstein et al. and Shermis and Burstein.14,15 For example, the rhetorical structure of a writing sample (its discourse elements: the ways a writer addresses an assignment’s purpose, audience, and subject) was not assessed.16 Nevertheless, the natural language grammar checker available in Microsoft Word allowed us to gather information about word count and the logic of sentence structure.


   Materials and Methods
Top
 Abstract
 The ability to write...
 The present study
 Materials and Methods
Results
 Discussion
 Conclusions
 References
 
In the freshman year at the New Jersey Dental School, all students complete a required writing assignment as part of their first semester course, Introduction to the Profession of Dentistry. Students are instructed to produce a 250–500 word essay on the portrayal of the profession in the popular media. It is emphasized to the students that this essay should be an example of their best-written work from both technical and intellectual perspectives. In other words, they are asked to write without error and to employ a kind of extended analytic thinking beyond mere summary.17

Beginning with the freshman class in the fall of 2001 and continuing with the freshman classes of 2002 and 2003, all written assignments in this course were required to be submitted electronically via the WebCTTM distance-learning platform.18 Students downloaded the essay question and instructions to a computer at the location of their choice (school, home, or other location), completed the required essay, and uploaded it to the course’s WebCT site. Submissions were required to be in HTML, enriched text, or Microsoft Word format; students were strongly encouraged to submit their essays as Word documents. Two weeks were provided for the students to complete the assignment. Since students were instructed that their essay should be an example of their best work, they were permitted to retract, replace, and resubmit their essays without penalty until midnight on the due date. Very few chose to replace their initial submissions. In cases where multiple submissions were made, only the final submission was included in the analysis.

Following the close of the submission period each year, a duplicate set of essays was created and the original set graded. Grades and comments were transmitted electronically to students. Essays submitted in text or HTML formats were converted to Microsoft Word documents. The duplicate set of essays was "cleaned" by a research assistant who removed all identifiers as well as page numbers, headers, footers, headings, subheading, and text enhancements. The outcome of this "cleaning" process was a uniform set of Microsoft Word documents, each consisting of text without any enhancements.

Using procedures available in Microsoft Word 2000, the following statistics were generated and recorded for each essay to evaluate surface features of the samples: counts including words, characters, paragraphs, and sentences; means (including sentences per paragraph); words per sentence; and characters per word.19 In addition, the following readability measures were generated and recorded to assess syntactic complexity: the percentage of passive sentences; readability as measured by the Flesch Reading Ease score; and language sophistication as measured by the Flesch-Kincaid Grade Level (both measures are described in the following paragraph). Although each essay was also checked for spelling and grammar, errors were not recorded because the technical quality of students’ submissions was beyond the scope of this project, which focused on testing the feasibility of conducting a computer-based assessment of students’ writing skills.

The Flesch Reading Ease score and the Flesch-Kincaid Grade Level scale are both widely used, reliable, and valid indicators of readability.20,21 The two instruments measure somewhat different aspects of readability using calculations based upon sentence length, number of sentences in a block of text, average number of syllables per word, and number of words per sentence. The "Help" files built into Microsoft Word 2000 provide the formulas used to calculate these scores.

Using a 100-point scale, the Flesch Reading Ease score rates text for ease of understanding. A score of 100 indicates text that is very easy to understand and 1 indicates that the text is very difficult to understand. The Flesch-Kincaid Grade Level score indicates the U.S. grade school level at which the text is written. Thus, a score of "6" would indicate that a sixth grader could read and understand the document.

To illustrate the meaning of and relationship between the Flesch Reading Ease score and the Flesch-Kincaid Grade Level, it is useful to consider how representative passages are scored. Applying these measures to the first paragraph of this article, for example, a Flesch Reading Ease score of 28.3 is derived. This indicates that the paragraph is difficult or complex to read and understand. The Flesch-Kincaid Grade Level instrument indicates that the same paragraph is written at a twelfth-grade level. This means that a twelfth grader should be able to read and understand the text. However, it should be noted that this instrument cannot indicate that text is written at higher than a twelfth-grade level and, thus, may actually underestimate the degree of reading skill necessary to read and interpret text. Thus, the first paragraph of this article requires a high level of skill to read and interpret, which may be interpreted as an indicator of the level of sophistication at which the authors write. To illustrate the other extreme, the authors developed the following passage modeled after a reading primer.

See Dick and Spot. See Spot run. Run Spot run.

See Sally and Puff. See Puff run. Run Puff run.

Sally plays with Puff. Dick watches Sally and Puff.

Sally and Dick grow up to be dentists.

Applying the Flesch Reading Ease instrument to this sample passage, a score of 100 is derived indicating that this passage is extremely easy to read. The Flesch-Kincaid Grade Level instrument rates this passage as 0.0 meaning that a beginning reader could read and interpret the text.

Applying the Flesch Reading Ease instrument and the Flesch-Kincaid Grade Level instrument to a sample of student writing yields valuable information. Kaplan et al. have demonstrated, for example, that essay length has a relatively large impact on the prediction of human ratings. Their study found that off-the-shelf computer-based writing assessment programs that use word counts predicted human ratings of essays exactly 60 percent of the time and within one score point 90 percent of the time. The researchers therefore concluded that essay length "continues to measure something appearing to be a proxy for component(s) of writing skills not directly measured by the grammar checkers."23 Surface features such as word count are thus not isolated features of writing and should not be dismissed as a ciphering of mere fluency and readability.24,25


   Results
Top
 Abstract
 The ability to write...
 The present study
 Materials and Methods
 Results
Discussion
 Conclusions
 References
 
Between the fall of 2001 and 2003, a total of 240 students (three consecutive first-year classes) completed the essay exercise on professional image. This represents virtually all students entering the dental school during this period. The exercise was completed by seventy-eight students in 2001, eighty-two students in 2002, and eighty students in 2003. In each of the entering classes, females outnumbered males. Overall, the study population was comprised of 132 females (55 percent) and 108 males (45 percent). There was no significant difference in gender composition among classes. Details of the size and gender composition of each class and of the overall study population are presented in Table 1Go.


View this table:
[in this window]
[in a new window]
  		Table 1. Sample size by year administered and gender
 
Students produced essays of varying lengths in spite of the instruction to keep lengths between 250 and 500 words (one to two pages, typed double-spaced). Although students were instructed to make this essay an example of their "best work" and had spelling and grammar checkers readily available, numerous errors were found in grammar, sentence structure, spelling, and punctuation. However, these errors were neither recorded nor analyzed since the assessment of specific technical performance was beyond the scope of this project.

Summary measures such as average number of sentences, average sentences per paragraph, and varied widely. The percentage of sentences written in passive voice showed particularly great variability, with percentages of passive sentences ranging from zero to 73 percent. Several measures showed significant differences among classes. Significant differences by class were found for total number of words per essay (image words), total number of characters per essay (image characters), total number of paragraphs per essay (image paragraphs), total number of sentences per essay (image sentences), average number of words per sentence (image average words sentence), and the percentage of passive sentences per essay (image percent passive). Table 2Go provides details of counts, summary measures, and the results of a one-way analysis of variance comparing the essays produced by the three consecutive first-year classes studied.


View this table:
[in this window]
[in a new window]
  		Table 2. One-way analysis of variance of essay structure
 
A separate analysis was performed to identify difference in performance between male and female students. This analysis failed to find any significant difference on any of the count or summary measures.

Readability measures were calculated and examined for the total population. In addition, readability measures were calculated and compared by class and by gender. The Flesch Reading Ease score for the entire population ranged from 14.6 to 74.0 with a mean of 50.561, indicating that the essays were moderately difficult to read. The essays produced by the freshman classes of 2002 and 2003 were somewhat more difficult to read than those produced by the class entering in 2001. This is clearly shown in the box plots comparing the performance of the three classes on this measure presented in Figure 1Go. These plots allow the comparison of medians and quartiles among the three groups. The box plots show that the median score for the entering class of 2001 was somewhat higher than the other groups, indicating that they produced essays that were somewhat easier (simpler and less complex) to read than those produced by the other two classes. In addition, an examination of the quartiles shows that there was greater variation among the members of the entering class of 2001 than among the other classes. A one-way analysis of variance was used to compare the Reading Ease scores among classes and confirmed that the observed differences were significant. Table 3Go provides the results of this analysis.



View larger version (10K):
[in this window]
[in a new window]
  		Figure 1. Flesch Reading Ease score for incoming dental school classes

 

View this table:
[in this window]
[in a new window]
  		Table 3. One-way analysis of variance of readability scores
 
The results of the Flesch-Kincaid Grade Level Analysis were similar to the results obtained for analysis of the Flesch Reading Ease scores. Overall, essays produced by the entire population were written at the tenth-grade level with the range extending from the sixth through the twelfth grades. The entering class of 2001 produced essays at the lowest grade level (10.009), with the entering classes of 2002 and 2003 producing essays that were written at a higher grade level (10.834 for the class entering in 2002 and 10.741 for the class entering in 2003). The one-way analysis of variance found that the Flesch Reading Ease score of the entering class of 2001 was significantly lower than the scores of the entering classes of 2002 and 2003. Box plots were also produced for this measure to allow a visual comparison on class performance. These plots (see Figure 2Go) demonstrate that the class of 2001 produced essays written at a lower grade level that the other two classes and that the range of variability among members of the class was somewhat greater than between the other two classes. These box plots also show a ceiling effect that is the result of the constricted range of the Flesch-Kincaid Grade Level measure that caps readability at the twelfth grade level.



View larger version (9K):
[in this window]
[in a new window]
  		Figure 2. Flesch-Kinkaid Grade Level analysis

 
The performance of male and female students was compared using both measures of readability. Neither instrument revealed any significant difference in performance on the image essay assignment.

The Flesch Reading Ease score and the Flesch-Kincaid Grade Level measure different but related aspects of readability. To examine the relationship between these two measures, the bivariate correlation (Pearson’s correlation) was calculated based on the results obtained from the overall population. The results obtained using these measures indicated a high negative correlation (–.834 with two tailed significance of p<.000). A negative correlation should be expected since polar scores indicate opposite levels of difficulty or grade level for the two instruments. Recall that a high Flesch-Kincaid Grade Level score indicates a higher level of reading skill necessary to understand a piece of written work, while a high score on the Flesch Reading Ease measure indicates that the written material is extremely easy to read. Thus, the results of these two measures should be negatively correlated as was found in this study.


   Discussion
Top
 Abstract
 The ability to write...
 The present study
 Materials and Methods
 Results
 Discussion
Conclusions
 References
 
The purpose of this study was to investigate the feasibility of using computer-based assessment to evaluate written work generated by dental students. Based on the results we obtained, computer-assisted evaluation appears to be feasible and produces useful results. These results support the recommendation of the National Commission on Writing in America’s School and Colleges that technology be applied to the evaluation of writing to overcome time and effort factors that inhibit the teaching of writing.

In this study, we tested the feasibility of evaluating the readability of student essays using two easily accessible instruments. Written assignments generated by three successive entering classes of dental students were evaluated using the Flesch Reading Ease score and the Flesch-Kincaid Grade Level indicator. These measures are included in virtually all versions of Microsoft Word, the de facto standard word processing program in most university settings. Though not the main purpose of the study, the data collected provide a useful baseline for assessing the performance of subsequent entering dental students as well as the future performance of these students as they move through dental school.

Assessment of the readability level of documents produced by students in their regular coursework was an efficient and effective mechanism for assessing the sophistication, though not the technical quality, of students’ written work. When coupled with the required electronic submission of written materials, the analysis of Reading Ease scores and Grade Level was straightforward, requiring only the grammar and spelling checker provided in Microsoft Word to calculate these measures and provide document statistics. These data can then be easily extracted and analyzed.

The automated identification of spelling and grammar errors provides the potential for supplying additional feedback to students regarding their performance. In addition, it can provide inputs for developing writing skills programs designed to address problem areas in written communication. Further, individualized data allows for the development of customized remediation programs for students with serious deficiencies in written communication skills.

These data suggest that entering dental students have a broad range of writing skills. Although the entrance standards on measures such as the Dental Achievement Test suggest great homogeneity among individuals applying to dental school on a number of variables, the scores from this study suggest considerable heterogeneity in students’ writing skills. Overall, the grade equivalent writing level of all three classes was below the eleventh grade comprehension level, with some students producing written material at a level of sophistication generally expected from middle school children. Although students were instructed to demonstrate their best work, Reading Ease scores suggested only a moderate level of sophistication in the work they produced.

The instruments used were sensitive to performance differences among classes when coupled with a basic analysis using box plots and one-way analysis of variance. Issues of sample size and privacy prevented the analysis of performance based on race, ethnicity, country of origin, or primary language of students. However, since the New Jersey Dental School is among the most culturally diverse in the nation, it is likely that these variables would have a measurable influence on performance on written tasks and should be assessed.

The Flesch Reading Ease score and the Flesch-Kincaid Grade Level indicator are readily available and easy to use since they are incorporated into widely used word processing software. However, numerous alternatives exist. Among the most widely used readability tests are the Gunning "Fog" Readability Test,26 the Fry Readability Graph,27 and the McLaughlin "Smog" formula.28 A limitation of several of the most widely used instruments, including the Flesch-Kincaid Grade Level instrument, is their inability to adequately deal with text written at higher than a twelfth-grade level. Further, formula-based readability measures only tell part of the story. As pointed out by Alice Horning,29 prepositional analysis, discourse analysis, and cohesion analysis using appropriate instruments are necessary to get a true indicator of readability. However, these analyses are more complex, not as readily accessible, and require a higher level of sophistication to interpret.


   Conclusions
Top
 Abstract
 The ability to write...
 The present study
 Materials and Methods
 Results
 Discussion
 Conclusions
References
 
The need for increased emphasis on student writing skills across the educational spectrum has been well documented. This study demonstrated the feasibility of applying readily available tools and techniques to the assessment of written materials produced by first-year dental students. The Flesch Reading Ease score and the Flesch-Kincaid Grade Level proved convenient to use, easy to interpret, and sensitive enough to identify performance difference among classes. Use of these instruments was facilitated by requiring students to prepare and submit written work electronically. Electronic submission of written materials by students also provides the opportunity for the assessment of the technical quality of student writing using standard spelling and grammar-checking software. Virtually all current dental students demonstrated sufficient computer skills to allow electronic submission to be a requirement. The use of a distance-learning platform such as WebCT facilitated the submission of student written work as well as the receipt of feedback by students.

In summary, the existing and widely available natural language assessment technology included in Microsoft Word shows promise as a vehicle for enhancing the assessment of dental students’ written communication skills. The ease of use and minimal training necessary to apply this technology can help mitigate the time-intensive nature of formal writing assessment.

As our study demonstrates, the combined use of WebCT technology (a system for electronically storing student work) and the Microsoft Word Flesch instruments (a system for electronically evaluating student work) meets the four criteria set by Kaplan et al. to ensure that the benefits of automated scoring exceed the costs.23 First, the information gained was defensible. The empirically established relationship between essay length and essay worth lends value to the information obtained in this study. Indeed, the great diversity in word length among the essays—even though a length was specified in the assignments—suggests that the homogeneity established in a set of DAT scores may not reflect the heterogeneity of that population when writing ability is considered. Such information is invaluable to administrators who must set curricular goals in the notoriously overcrowded first two years of dental school. In that writing has long been established as a way to enable students to better understand their disciplines, the importance of helping students gain stronger writing skills early in their dental careers cannot be emphasized enough. As Sternglass reported based upon a longitudinal study of college level writing, "As the writing and conceptual development of the students in the study reveal, over several semesters and even several years, through consistent instructional prodding both in writing and discipline area courses, students can develop an analytic stance that permits them to understand the significance of ideas in the particular field to the level where they become able to question some of the assumptions in that field."30 It would serve the dental profession well to consider these findings during the next review of the Commission on Dental Accreditation’s curriculum requirements. We advocate the inclusion of the teaching of writing in the curriculum guidelines.

Second, the information collected was accurate, and third, the evaluation method is not coachable. Machine evaluation of surface features (those aspects of writing such as word count that require no inference) is consistent and reliable. While it could be imagined that a student would simply paste into WebCT multiple copies of the material to increase word count, such a technique would easily be identified by the human reader assigning an individual grade and comment to the reader. As Powers et al. reported, even deliberate attempts by sophisticated writers to stump automated systems are inherently difficult.31 In their study of attempts to stump E-Rater, the one reader who successfully stumped the program was a professor of computational linguistics who wrote several paragraphs and repeated them thirty-seven times. However, as the researchers concluded, such a technique would easily be identified and discounted by a human reader.

Fourth, the evaluation method is cost effective. Although some time was required to transfer the student writing samples from WebCT to Microsoft Word, this opportunity cost was not prohibitive to the present study.

There is, of course, much more work to be undertaken if Kaplan’s four criteria are to be fully met. The information is defensible, but the face validity of machine scoring for surface features, however highly correlated that information may be with measures designed to capture total ability, leaves much to be desired. While the information gained in this study was accurate, further study presently being undertaken by the authors is needed to assess the writing ability of dental students on variables not included in the present study. A student’s ability to think critically regarding the rhetorical specifications of an assignment—the way the student addresses the purpose, audience, and subject of an assignment—can best be assessed through multiple observations of student writing contained in portfolios.32 Hence, in increasing our accuracy with further study, we can increase the defensibility of this methodology. In addition, in that the prohibitive, time-consuming cost of portfolio scoring is well known, the relationship between portfolio scores and the Flesh scores may identify ways to use cost-efficient, computer-based scores of surface features in conjunction with human-based portfolio scores of rhetorical ability. If such assessment information is then used to enhance instruction, the distance between assessment and instruction may be more readily bridged through an increase in the use of technology.


   Footnotes
 
Dr. Holtzman is Professor, Department of Community Health, University of Medicine and Dentistry of New Jersey-New Jersey Dental School; Dr. Elliot is Professor of English, Department of Humanities, New Jersey Institute of Technology and Adjunct Professor, Department of Community Health, University of Medicine and Dentistry of New Jersey-New Jersey Dental School; Dr. Biber is Associate Professor, Department of Restorative Dentistry and Office of Academic Affairs, University of Medicine and Dentistry of New Jersey-New Jersey Dental School; and Dr. Sanders is Associate Professor and Director of Patient Care Services, University of Nevada, Las Vegas, School of Dental Medicine. Direct correspondence to Dr. Joseph Holtzman, Department of Community Health, UMDNJ-NJDS, 110 Bergen Street, Room B-848, Newark, New Jersey 07101-1709; 973-972-7133 phone; 973-972-0363 fax; holtzman{at}umdnj.edu.


   REFERENCES
Top
 Abstract
 The ability to write...
 The present study
 Materials and Methods
 Results
 Discussion
 Conclusions
 References
 

  1. Graff HJ. The literacy myth: cultural integration and social structure in the nineteenth century. New Brunswick, NJ: Transaction Publishers, 1991.
  2. Resnick DP, Resnick AB. The nature of literacy: an historical exploration. Harvard Educ Rev 1977;47(3):370–85.
  3. Huot B. (Re)articulating writing assessment for teaching and learning. Logan: Utah State University Press, 2002.
  4. Snyder TD. 120 years of American education: a statistical portrait. Washington, DC: National Center for Education Statistics, 1993.
  5. The neglected "R": the need for a writing revolution. Report of the National Commission on Writing in America’s Schools and Colleges. New York: The College Board, 2003.
  6. Projections of education statistics to 2013. Washington, DC: National Center for Education Statistics, 2003.
  7. Ong W. Orality and literacy: the technologizing of the word. London: Methuen, 1981.
  8. McAndrew R, Potts AJ, McAndrew M, Adam S. Opinions of dental consultants on the standard of referral letters in dentistry. Br Dent J 1997;182(1):22–5.[Medline]
  9. Leith R, Lowry L, O’Sullivan M. Communication between dentists and laboratory technicians. J Ir Dent Assoc 2000;46(1):5–10.[Medline]
  10. Alexander RE. Patient understanding of postsurgical instruction forms. Oral Surg Oral Med Oral Pathol Oral Radiol Endod 1999;87(2):153–8.[Medline]
  11. Academic literacy: a statement of competencies expected of students entering California’s public colleges and universities. Sacramento: Intersegmental Committee of the Academic Senates of the California Community Colleges, the California State University, and the University of California, 2003.
  12. Bartlett T. Why Johnny can’t write, even though he went to Princeton. Chron Higher Educ 2003;49(17):A39.
  13. Bjork RE. The dentist is writing. J Dent Educ 1982;46(11):639–43.[Abstract]
  14. Burstein JC, Braden-Harder L, Chodorow M, Kaplan BA, Kukich K, Lu C, et al. Computer analysis of essay content for automated score prediction: a prototype automated scoring system for GMAT analytic writing assessment essays. ETS Research Report 98-15. Princeton: Educational Testing Service, 1998.
  15. Shermis MD, Burstein J. Automated essay scoring: a cross-disciplinary perspective. Hillsdale, NJ: Lawrence Erlbaum, 2003.
  16. Brossell G. Current research and unanswered questions in writing assessment. In: Greenberg KL, Wiener HS, Donovan RS, eds. Writing assessment: issues and strategies. New York: Longman, 1986:168–82, esp. 173.
  17. Murphy RL, Murphy S. Designing writing tasks for the assessment of writing. Norwood, NJ: Ablex, 1998.
  18. WebCTTM Campus Edition 4.1.
  19. Microsoft® 2000 (9.0.3821 SR-1).
  20. Flesch R. A readability formula that saves time. J Reading 1968;11(7):513–6
  21. Paache-Orlow MK, Taylor HA, Brancati FL. Readability standards for informed-consent forms as compared with actual readability. N Engl J Med 2003;348(8): 721–6.[Abstract/Free Full Text]
  22. Kaplan RM, Wolff S, Burstein J, Lu C, Rock D, Kaplan B. Scoring essays automatically using surface features. ETS Research Report 98-39. Princeton: Educational Testing Service, 1998.
  23. Hunt KW. Early blooming and late blooming syntactic structures. In: Cooper CR, Odell L, eds. Evaluating writing: describing, measuring, judging. Urbana, IL: NCTE, 1977:91–104.
  24. Page EB, Petersen N. The computer moves into essay grading: updating the ancient test. Phi Delta Kappan 1995(March):561–5.
  25. Gunning R. The technique of clear writing. New York: McGraw-Hill, 1952.
  26. Fry E. Elementary reading instruction. New York: McGraw Hill, 1977.
  27. McLaughlin H. SMOG grading: a new readability formula. J Reading 1969;12(8):639–46.
  28. Horning AS. Readability: reading/writing tools for measurement. J Adv Composition 1987;7:101–11.
  29. Sternglass MS. Time to know them: a longitudinal study of writing and learning at the college level. Mahwah, NJ: Lawrence Erlbaum, 1997.
  30. Powers DE, Burstein J, Chodorow M, Fowles ME, Kukich K. Stumping E-Rater: challenging the validity of automated essay scoring. ETS Research Report 01-03. Princeton: Educational Testing Service, 2001.
  31. Black L, Daiker DA, Sommers J, Stygall G, eds. New directions in portfolio assessment: reflective practice, critical theory, and large-scale scoring. Portsmouth, NH: Heinemann-Boynton/Cook, 1994.




This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Holtzman, J. M.
Right arrow Articles by Sanders, R. M.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Holtzman, J. M.
Right arrow Articles by Sanders, R. M.

HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS