Machine Scoring of Student Essays
Truth and Consequences
Publication Year: 2006
The current trend toward machine-scoring of student work, Ericsson and Haswell argue, has created an emerging issue with implications for higher education across the disciplines, but with particular importance for those in English departments and in administration. The academic community has been silent on the issue—some would say excluded from it—while the commercial entities who develop essay-scoring software have been very active.
Machine Scoring of Student Essays is the first volume to seriously consider the educational mechanisms and consequences of this trend, and it offers important discussions from some of the leading scholars in writing assessment.
Reading and evaluating student writing is a time-consuming process, yet it is a vital part of both student placement and coursework at post-secondary institutions. In recent years, commercial computer-evaluation programs have been developed to score student essays in both of these contexts. Two-year colleges have been especially drawn to these programs, but four-year institutions are moving to them as well, because of the cost-savings they promise. Unfortunately, to a large extent, the programs have been written, and institutions are installing them, without attention to their instructional validity or adequacy.
Since the education software companies are moving so rapidly into what they perceive as a promising new market, a wider discussion of machine-scoring is vital if scholars hope to influence development and/or implementation of the programs being created. What is needed, then, is a critical resource to help teachers and administrators evaluate programs they might be considering, and to more fully envision the instructional consequences of adopting them. And this is the resource that Ericsson and Haswell are providing here.
Published by: Utah State University Press
Cover
Title Page, Copyright
CONTENTS
Download PDF (70.7 KB)
pp. v-vi
INTRODUCTION
Download PDF (84.0 KB)
pp. 1-7
We’re in the fifth year of the twenty-first century and the Parliament of India, several universities in Italy, and four Catholic churches in Monterrey, Mexico, all have bought cell-phone jammers. Meanwhile in the State of Texas, USA, the State Board of Education has decided that students who fail the essay-writing ...
1 INTERESTED COMPLICITIES: The Dialectic of Computer-Assisted Writing Assessment
Download PDF (133.2 KB)
pp. 8-27
She knew how difficult creating something new had proved. And she certainly had learned the hard way that there were no easy shortcuts to success. In particular, she remembered with embarrassment how she had tried to crash through the gates of success with a little piece on a young author struggling to succeed, ...
2 THE MEANING OF MEANING: Is a Paragraph More than an Equation?
Download PDF (105.8 KB)
pp. 28-37
Several chapters in this collection allude to or deal briefly with issues of “meaning” in the controversy about the machine scoring of essays. This chapter’s intent is to explore extensively the “meaning of meaning,” arguing that, although they may appear to be esoteric, considerations of “meaning” are central to the controversy ...
3 CAN’T TOUCH THIS: Reflections on the Servitude of Computers as Readers
Download PDF (129.7 KB)
pp. 38-56
Consider, for a moment, what’s going on. First, you’re in a multidimensional context where you and I, and this text, share a presence, a purpose, and knowledge that delimit the interpretive possibilities and let you begin fitting into boxes what little you’ve seen so far, or maybe shaping a box around it: academic genre, ...
4 AUTOMATONS AND AUTOMATED SCORING: Drudges, Black Boxes, and Dei Ex Machina
Download PDF (139.7 KB)
pp. 57-78
Her name really is Nancy Drew. Like her fictional namesake, she is into saving people, although more as the author of a mystery than the hero of one. She teaches English at a high school in Corpus Christi, Texas, and according to the local newspaper (Beshur 2004), she has designed a software program that will grade student essays. ...
5 TAKING A SPIN ON THE INTELLIGENT ESSAY ASSESSOR
Download PDF (103.3 KB)
pp. 79-92
The following narrative recounts an experiment I performed upon a particular essay-scoring machine, the Intelligent Essay Assessor (IEA), that was first brought to my attention by Anne Herrington and Charles Moran’s 2001 College English essay “What Happens When Machines Read Our Students’ Writing?” ...
6 ACCUPLACER’S ESSAY-SCORING TECHNOLOGY: When Reliability Does Not Equal Validity
Download PDF (135.6 KB)
pp. 93-113
Placement of students in first-year writing courses is generally seen as a time-consuming but necessary exercise at most colleges and universities in the United States. Administrators have been concerned about both the expense and inconvenience of testing, about the validity of the tests, and about the reliability of the scorers. ...
7 WRITEPLACER PLUS IN PLACE: An Exploratory Case Study
Download PDF (119.2 KB)
pp. 114-129
In 2001, we published an essay in College English entitled “What Happens When Machines Read Our Students’ Writing?” In it, we discussed two computer programs, then relatively new to the market, that were designed to evaluate student writing automatically: WritePlacer Plus, developed by Vantage Technology, ...
8 E-WRITE AS A MEANS FOR PLACEMENT INTO THREE COMPOSITION COURSES: A Pilot Study
Download PDF (99.3 KB)
pp. 130-137
In the fall of 2002 Utah Valley State College (UVSC) began institutional research into placement tests for first-year composition courses: two basic writing courses and a freshman composition course. UVSC researchers had previously presented evidence in the article “Basic Writing Placement with Holistically Scored ...
9 COMPUTERIZED WRITING ASSESSMENT: Community College Faculty Find Reasons to Say “Not Yet”
Download PDF (100.7 KB)
pp. 138-146
Community colleges exist to provide educational opportunities to a fluid population, many of whom encounter sudden changes in their work, family lives, and financial situations. For this reason, community colleges often admit, place, and register a student for classes all within a little more than twenty-four hours. ...
10 PILOTING THE COMPASS E-WRITE SOFTWARE AT JACKSON STATE COMMUNITY COLLEGE
Download PDF (84.2 KB)
pp. 147-153
Placement issues are a major concern in higher education. Many states require students who do not have college-level scores on entrance exams such as the ACT or SAT to take precollege developmental classes. Without reliable placement testing, students may be put into classes that are too easy for them and become bored, ...
11 THE ROLE OF THE WRITING COORDINATOR IN A CULTURE OF PLACEMENT BY ACCUPLACER
Download PDF (97.7 KB)
pp. 154-165
Placement processes into college writing and developmental writing courses include diverse options. Processes include use of external exams and externally set indicators, such as SAT or ACCUPLACER scores; use of locally designed essay topics for placement that are assessed by human readers using either holistic ...
12 ALWAYS ALREADY: Automated Essay Scoring and Grammar-Checkers in College Writing Courses
Download PDF (105.9 KB)
pp. 166-176
Although Ken S. McAllister and Edward M. White call the development of automated essay scoring “a complex evolution driven by the dialectic among researchers, entrepreneurs, and teachers” (chapter 1 of this volume), within composition studies the established tradition points toward the rejection of ...
13 AUTOMATED ESSAY GRADING IN THE SOCIOLOGY CLASSROOM: Finding Common Ground
Download PDF (138.8 KB)
pp. 177-198
This chapter describes an effort by one author, a sociologist, to introduce automated essay grading in the classroom, and the concerns raised by the other author, the director of a campuswide writing program, in evaluating the grading scheme for fulfillment of a writing-intensive (WI) requirement. ...
14 AUTOMATED WRITING INSTRUCTION: Computer-Assisted or Computer-Driven Pedagogies?
Download PDF (110.6 KB)
pp. 199-210
Elsewhere in this collection William Condon (chapter 15) exposes the losses college writing programs may experience when employing machine scoring in the assessment process. Contrasting machine scoring with a host of other more “robust” forms of assessment, such as portfolio-based assessment, ...
15 WHY LESS IS NOT MORE: What We Lose by Letting a Computer Score Writing Samples
Download PDF (105.2 KB)
pp. 211-220
Earlier in this volume, Rich Haswell (chapter 4) questions the validity of machine scoring by tying it to holistic scoring methodologies—and I certainly agree with his critique of timed writings, holistically scored. However, I want to suggest that questions about machine scoring differ from questions about holistic readings. ...
16 MORE WORK FOR TEACHER? Possible Futures of Teaching Writing in the Age of Computerized Assessment
Download PDF (112.2 KB)
pp. 221-233
In her book More Work for Mother: The Ironies of Household Technology from the Open Hearth to the Microwave (1983), Ruth Schwartz Cowan presents a feminist history of modern household technology. As the title of her book emphasizes, her argument is that the hundreds of “gadgets” invented with the purpose of easing ...
17 A BIBLIOGRAPHY OF MACHINE SCORING OF STUDENT WRITING, 1962–2005
Download PDF (102.9 KB)
pp. 234-243
This bibliography focuses on the theory, design, application, and implications of automated or machine rating of extended student writing. It covers the literature in English that speculates and reports on the success of computers in scoring “free text” or essaylike compositions of students. ...
GLOSSARY OF TERMS
Download PDF (73.3 KB)
pp. 244-245
NOTES
Download PDF (84.4 KB)
pp. 246-250
REFERENCES
Download PDF (108.5 KB)
pp. 251-261
INDEX
Download PDF (81.3 KB)
pp. 262-266
CONTRIBUTORS
E-ISBN-13: 9780874215366
Print-ISBN-13: 9780874216325
Publication Year: 2006
