Mark. D. Shermis, dean of the College of Education at the University of Akron, who led a study comparing human graders to software designed to score student essays, said this past April that robo-graders built to evaluate student writing may be more reliable than human graders.

Other researchers, however, disagree; Les Perelman, former director of writing and current research affiliate at the Massachussetts Institute of Technology, contends in an editorial published in the Boston Globe, “Robo-graders do not score by understanding meaning but almost solely by use of gross measures, especially length and the presence of pretentious language. The fallacy underlying this approach is confusing association with causation. A person makes the observation that many smart professors wear tweed jackets and then believes that if she wears a tweed jacket, she will be a smart college professor.”

Perelman goes onto argue that private vendors and researchers that produce robo-readers aren’t transparent enough about the methods of robo-graders. “None of the major testing companies allow easy access or open-ended demonstrations of their robo-graders,” he writes.

However, according to writer Annie Murphy Paul at The HERRINGER REPORT last week, robo-readers, rather than robo-graders, may encourage students to revise more often and subsequently better their writing skills by offering neutral, rather than “corrective” or “punitive” criticism. An independent study that that looks at the effects of robo-reader Criterion on the writing of students learning to teach English as a second language revealed that student writing significantly improved; writers repeated less words, used shorter, simpler sentences, and edited grammar and spelling errors.

Other robo-readers and graders include e-Rater and Pearson EssayScorer, and the latter is now in use at Bristow Middle School in Brentwood, California.