The Museum of Hoaxes
hoax archive hoax archive hoax archive hoax archive hoax archive
Hoaxes Throughout History
Middle AgesEarly Modern1700s1800-1840s1850-1890s
1900s1910s1920s1930s1940s1950s1960s1970s1980s1990s21st Century2014
Computer Grading?
Robert Yagelowich pointed this article out to me, and like him what I read initially made me pretty skeptical. The article describes a computer program that's being used to grade student essays. Not just grade the spelling and punctuation, but the content itself. Since computers can't even be relied on to spellcheck very well, I couldn't imagine how they would grade content. I had suspicions of another ChatNannies type of hoax. But apparently computer-grading is real. The software, called E-Rater, has been developed by Educational Testing Services, and they provide an online demo of how it works. I used to be a teaching assistant at UC San Diego, and I graded thousands of student essays. And I have to admit that human graders are often pretty fallible. By the time you get to the bottom of a stack of essays, you're just going through those things as fast as possible, barely reading them. So maybe a computer could grade essays better than a human. The computer, at least, wouldn't grade differently depending on how much coffee it had drunk, or how little sleep it had got.
Posted by The Curator on Thu May 20, 2004

I'm wondering if the 'demo' is a hoax... there's nothing in it that actually 'demos' the product.
Posted by Bob S. Yunkle  on  Fri May 21, 2004  at  12:42 PM
Good point Bob

the buttons just use javascript to open html pages with the "results"
Posted by Peter  in  London  on  Fri May 21, 2004  at  04:29 PM
A lot of employers use similar programs to evaluate the resumes of prospective employers, so I guess it's not implausible. A lot of profs grade essays based on a "hit list," anyway--checking off mentions of key concepts with little regard to composition or original insight.
Posted by James Joyner  on  Sun May 23, 2004  at  10:28 AM
I agree with Bob that there is a failure to demonstration the product. The fact that the product only seems to require the topic of the essay to judge quality is unsettling since there is no way that a program can derive meaning from all possible contexts and language patterns. What is shown by the "demo" isn't impressive anyway. The topic as well as the work itself was simplistic. The authors of the essays used terms from the topic--"what they want to do" and "what they feel they should do" in almost every instance. That's standardized testing style writing--poor and repetitive. I doubt this program, if it even exists, could grade anything past a 5th grade end of year state exam (and manage to fail the most creative students with the most elegant prose while it's at it).
Posted by Mary  on  Fri Jul 02, 2004  at  10:36 AM
Commenting is no longer available in this channel entry.
All text Copyright © 2014 by Alex Boese, except where otherwise indicated. All rights reserved.