Sting Operation': The Stunning Percentage of Science Journals That Accepted a Completely Bogus Study

Science magazine wanted to figure out just how legitimate open-access, peer-reviewed journals are. So, it set out to dupe them with a completely fake study.

The sting operation conducted by Science magazine showed many open-access journals lack the quality control to catch fake or faulty science -- or even ignore peer reviews pointing it out -- in order to make profit on submissions. (Image source: Shutterstock.com)

What it found is surprising some in the scientific industry, while only confirming the fears of others. More than half of the journals John Bohannon submitted his paper about the fictitious, anticancer properties identified in a lichen compound were accepted for publication.

The first and easiest clue that could have been picked out by the journals was that the study's author, Ocorrafoo Cobange, does not exist as a real person, nor does his research institute, the Wassee Institute of Medicine.

But beyond that, Bohannon wrote in Science that "any reviewer with more than a high-school knowledge of chemistry and the ability to understand a basic data plot should have spotted the paper's short-comings immediately."

"Its experiments are so hopelessly flawed that the results are meaningless," Bohannon continued.

The experiment was testing so-called "open-access" journals -- those that are not subscription based. There are some who advocate in favor of open-access to help spread scientific knowledge, while others want the system to remain remain subscription-oriented, which can create a financial barrier.

Of the 304 submissions of the fake study during a 10-month timeframe (only 255 submitted received some sort of response from editors) 157 seemed to miss the study's "fatal flaws" in the "sting operation" that Bohannan said shows "the contours of an emerging Wild West in academic publishing."

Sixty percent of this 255 didn't undergo any peer review, which for those that were rejected is "good news," according to Bohannan, because it means editors didn't even waste reviewers time with it. For those that accepted it without peer review it means "the paper was rubber-stamped without being read by anyone."

Of the 106 submissions that did undergo review, only 36 recognized the scientific problems with the study. Sixteen publications, even with "damning reviews," still accepted the paper.

Here's more from Bohannan's perspective on what the sting revealed.(emphasis added):

From humble and idealistic beginnings a decade ago, open-access scientific journals have mushroomed into a global industry, driven by author publication fees rather than traditional subscriptions. Most of the players are murky. The identity and location of the journals' editors, as well as the financial workings of their publishers, are often purposefully obscured. But Science's investigation casts a powerful light. Internet Protocol (IP) address traces within the raw headers of e-mails sent by journal editors betray their locations. Invoices for publication fees reveal a network of bank accounts based mostly in the developing world. And the acceptances and rejections of the paper provide the first global snapshot of peer review across the open-access scientific enterprise.

[...]

Acceptance was the norm, not the exception. The paper was accepted by journals hosted by industry titans Sage and Elsevier. The paper was accepted by journals published by prestigious academic institutions such as Kobe University in Japan. It was accepted by scholarly society journals. It was even accepted by journals for which the paper's topic was utterly inappropriate, such as the Journal of Experimental & Clinical Assisted Reproduction.

What it seems to come down to is profit.

"But even when editors and bank accounts are in the developing world, the company that ultimately reaps the profits may be based in the United States or Europe. In some cases, academic publishing powerhouses sit at the top of the chain," Bohannan wrote.

Another interesting find was that an open-access journal that Bohannan said is among the many that have been "criticized for poor quality control" actually had the "most rigorous peer review of all." The journal PLOS One, for example, was the only one pointing out some of the study's ethical issues. This journal rejected the fictitious paper due to poor scientific quality.

Read about the full investigation in Bohannan's full article in Science.

Featured image via Shutterstock.com.

--

[related]