We cannot verify your location
Browse Book and Journal Content on Project MUSE
OR
title

Machine Scoring of Student Essays

Truth and Consequences

edited by Patricia Freitag Ericsson & Rich Haswell

Publication Year: 2006

The current trend toward machine-scoring of student work, Ericsson and Haswell argue, has created an emerging issue with implications for higher education across the disciplines, but with particular importance for those in English departments and in administration. The academic community has been silent on the issue—some would say excluded from it—while the commercial entities who develop essay-scoring software have been very active.

Machine Scoring of Student Essays is the first volume to seriously consider the educational mechanisms and consequences of this trend, and it offers important discussions from some of the leading scholars in writing assessment.

Reading and evaluating student writing is a time-consuming process, yet it is a vital part of both student placement and coursework at post-secondary institutions. In recent years, commercial computer-evaluation programs have been developed to score student essays in both of these contexts. Two-year colleges have been especially drawn to these programs, but four-year institutions are moving to them as well, because of the cost-savings they promise. Unfortunately, to a large extent, the programs have been written, and institutions are installing them, without attention to their instructional validity or adequacy.

Since the education software companies are moving so rapidly into what they perceive as a promising new market, a wider discussion of machine-scoring is vital if scholars hope to influence development and/or implementation of the programs being created. What is needed, then, is a critical resource to help teachers and administrators evaluate programs they might be considering, and to more fully envision the instructional consequences of adopting them. And this is the resource that Ericsson and Haswell are providing here.

Published by: Utah State University Press

Title Page, Copyright

pdf iconDownload PDF (43.2 KB)
 

CONTENTS

pdf iconDownload PDF (70.7 KB)
pp. v-vi

read more

INTRODUCTION

pdf iconDownload PDF (84.0 KB)
pp. 1-7

We’re in the fifth year of the twenty-first century and the Parliament of India, several universities in Italy, and four Catholic churches in Monterrey, Mexico, all have bought cell-phone jammers. Meanwhile in the State of Texas, USA, the State Board of Education has decided that students who fail the essay-writing ...

read more

1 INTERESTED COMPLICITIES: The Dialectic of Computer-Assisted Writing Assessment

pdf iconDownload PDF (133.2 KB)
pp. 8-27

She knew how difficult creating something new had proved. And she certainly had learned the hard way that there were no easy shortcuts to success. In particular, she remembered with embarrassment how she had tried to crash through the gates of success with a little piece on a young author struggling to succeed, ...

read more

2 THE MEANING OF MEANING: Is a Paragraph More than an Equation?

pdf iconDownload PDF (105.8 KB)
pp. 28-37

Several chapters in this collection allude to or deal briefly with issues of “meaning” in the controversy about the machine scoring of essays. This chapter’s intent is to explore extensively the “meaning of meaning,” arguing that, although they may appear to be esoteric, considerations of “meaning” are central to the controversy ...

read more

3 CAN’T TOUCH THIS: Reflections on the Servitude of Computers as Readers

pdf iconDownload PDF (129.7 KB)
pp. 38-56

Consider, for a moment, what’s going on. First, you’re in a multidimensional context where you and I, and this text, share a presence, a purpose, and knowledge that delimit the interpretive possibilities and let you begin fitting into boxes what little you’ve seen so far, or maybe shaping a box around it: academic genre, ...

read more

4 AUTOMATONS AND AUTOMATED SCORING: Drudges, Black Boxes, and Dei Ex Machina

pdf iconDownload PDF (139.7 KB)
pp. 57-78

Her name really is Nancy Drew. Like her fictional namesake, she is into saving people, although more as the author of a mystery than the hero of one. She teaches English at a high school in Corpus Christi, Texas, and according to the local newspaper (Beshur 2004), she has designed a software program that will grade student essays. ...

read more

5 TAKING A SPIN ON THE INTELLIGENT ESSAY ASSESSOR

pdf iconDownload PDF (103.3 KB)
pp. 79-92

The following narrative recounts an experiment I performed upon a particular essay-scoring machine, the Intelligent Essay Assessor (IEA), that was first brought to my attention by Anne Herrington and Charles Moran’s 2001 College English essay “What Happens When Machines Read Our Students’ Writing?” ...

read more

6 ACCUPLACER’S ESSAY-SCORING TECHNOLOGY: When Reliability Does Not Equal Validity

pdf iconDownload PDF (135.6 KB)
pp. 93-113

Placement of students in first-year writing courses is generally seen as a time-consuming but necessary exercise at most colleges and universities in the United States. Administrators have been concerned about both the expense and inconvenience of testing, about the validity of the tests, and about the reliability of the scorers. ...

read more

7 WRITEPLACER PLUS IN PLACE: An Exploratory Case Study

pdf iconDownload PDF (119.2 KB)
pp. 114-129

In 2001, we published an essay in College English entitled “What Happens When Machines Read Our Students’ Writing?” In it, we discussed two computer programs, then relatively new to the market, that were designed to evaluate student writing automatically: WritePlacer Plus, developed by Vantage Technology, ...

read more

8 E-WRITE AS A MEANS FOR PLACEMENT INTO THREE COMPOSITION COURSES: A Pilot Study

pdf iconDownload PDF (99.3 KB)
pp. 130-137

In the fall of 2002 Utah Valley State College (UVSC) began institutional research into placement tests for first-year composition courses: two basic writing courses and a freshman composition course. UVSC researchers had previously presented evidence in the article “Basic Writing Placement with Holistically Scored ...

read more

9 COMPUTERIZED WRITING ASSESSMENT: Community College Faculty Find Reasons to Say “Not Yet”

pdf iconDownload PDF (100.7 KB)
pp. 138-146

Community colleges exist to provide educational opportunities to a fluid population, many of whom encounter sudden changes in their work, family lives, and financial situations. For this reason, community colleges often admit, place, and register a student for classes all within a little more than twenty-four hours. ...

read more

10 PILOTING THE COMPASS E-WRITE SOFTWARE AT JACKSON STATE COMMUNITY COLLEGE

pdf iconDownload PDF (84.2 KB)
pp. 147-153

Placement issues are a major concern in higher education. Many states require students who do not have college-level scores on entrance exams such as the ACT or SAT to take precollege developmental classes. Without reliable placement testing, students may be put into classes that are too easy for them and become bored, ...

read more

11 THE ROLE OF THE WRITING COORDINATOR IN A CULTURE OF PLACEMENT BY ACCUPLACER

pdf iconDownload PDF (97.7 KB)
pp. 154-165

Placement processes into college writing and developmental writing courses include diverse options. Processes include use of external exams and externally set indicators, such as SAT or ACCUPLACER scores; use of locally designed essay topics for placement that are assessed by human readers using either holistic ...

read more

12 ALWAYS ALREADY: Automated Essay Scoring and Grammar-Checkers in College Writing Courses

pdf iconDownload PDF (105.9 KB)
pp. 166-176

Although Ken S. McAllister and Edward M. White call the development of automated essay scoring “a complex evolution driven by the dialectic among researchers, entrepreneurs, and teachers” (chapter 1 of this volume), within composition studies the established tradition points toward the rejection of ...

read more

13 AUTOMATED ESSAY GRADING IN THE SOCIOLOGY CLASSROOM: Finding Common Ground

pdf iconDownload PDF (138.8 KB)
pp. 177-198

This chapter describes an effort by one author, a sociologist, to introduce automated essay grading in the classroom, and the concerns raised by the other author, the director of a campuswide writing program, in evaluating the grading scheme for fulfillment of a writing-intensive (WI) requirement. ...

read more

14 AUTOMATED WRITING INSTRUCTION: Computer-Assisted or Computer-Driven Pedagogies?

pdf iconDownload PDF (110.6 KB)
pp. 199-210

Elsewhere in this collection William Condon (chapter 15) exposes the losses college writing programs may experience when employing machine scoring in the assessment process. Contrasting machine scoring with a host of other more “robust” forms of assessment, such as portfolio-based assessment, ...

read more

15 WHY LESS IS NOT MORE: What We Lose by Letting a Computer Score Writing Samples

pdf iconDownload PDF (105.2 KB)
pp. 211-220

Earlier in this volume, Rich Haswell (chapter 4) questions the validity of machine scoring by tying it to holistic scoring methodologies—and I certainly agree with his critique of timed writings, holistically scored. However, I want to suggest that questions about machine scoring differ from questions about holistic readings. ...

read more

16 MORE WORK FOR TEACHER? Possible Futures of Teaching Writing in the Age of Computerized Assessment

pdf iconDownload PDF (112.2 KB)
pp. 221-233

In her book More Work for Mother: The Ironies of Household Technology from the Open Hearth to the Microwave (1983), Ruth Schwartz Cowan presents a feminist history of modern household technology. As the title of her book emphasizes, her argument is that the hundreds of “gadgets” invented with the purpose of easing ...

read more

17 A BIBLIOGRAPHY OF MACHINE SCORING OF STUDENT WRITING, 1962–2005

pdf iconDownload PDF (102.9 KB)
pp. 234-243

This bibliography focuses on the theory, design, application, and implications of automated or machine rating of extended student writing. It covers the literature in English that speculates and reports on the success of computers in scoring “free text” or essaylike compositions of students. ...

GLOSSARY OF TERMS

pdf iconDownload PDF (73.3 KB)
pp. 244-245

NOTES

pdf iconDownload PDF (84.4 KB)
pp. 246-250

REFERENCES

pdf iconDownload PDF (108.5 KB)
pp. 251-261

INDEX

pdf iconDownload PDF (81.3 KB)
pp. 262-266

CONTRIBUTORS

pdf iconDownload PDF (65.9 KB)
 


E-ISBN-13: 9780874215366
Print-ISBN-13: 9780874216325

Publication Year: 2006