Open-Access Hoax

A report of a scientific hoax appears in the latest issue of Science. Researcher John Bohannon wrote a purposefully bad scientific paper — one with glaringly bad errors that any peer reviewer should be able to spot. He then submitted versions of that paper to 304 open-access journals, using aliases such as "Ocarrafoo M.L. Cobange," supposedly a biologist at the Wassee Institute of Medicine in Asmara. The result: "More than half of the journals accepted the paper, failing to notice its fatal flaws."

Bohannon says this raises "questions about peer-review practices in much of the open-access world."

But defenders of the open-access system (such as here and here) point out that Bohannon didn't submit the article to any subscription-based journals, which makes it impossible to know whether his experiment shows something uniquely bad about the open-access system, or whether it reveals a problem endemic in peer review as a whole.

Bohannon's experiment recalls a similar one conducted by Douglas Peters and Stephen Ceci back in 1980, in which they took ten already published scientific articles, changed the name of the author and title of the article, and then resubmitted the articles to the same journal in which they had previously been published. The majority of editors rejected the articles without recognizing they had already been published in their own journal.


Posted on Sat Oct 05, 2013


Another major problem with Bohannon's experiment is that he chose journals in equal numbers from both the Directory of Open Access Journals which lists all the best ones and Bealle's List of Predatory Publishers which lists all the worst. That's a rather stacked deck towards making OA look bad. Any minor peer review of Bohannon's Science article would have seen that, but it didn't get any.

Twenty-five percent of the good list accepted the paper which is a bit worrying and around fifteen percent of the bad list rejected it.
Posted by David Shenk  on  Tue Oct 15, 2013  at  08:16 AM
Commenting is not available in this channel entry.