How long has peer review been in crisis? At what point does crisis outlast emergency to become status quo? Attacks on the weaknesses of peer review appear with such regularity that they have migrated from scholarly journals to newspapers and magazines. Notwithstanding criticism—and bold experiments such as the experimental open peer review given online to Kathleen Fitzpatrick’s 2011 book Planned Obsolescence before its publication—the gears grind on, due in large part to the reward systems built around the mechanism of blind and anonymous review.
Among those tilting at the windmill of reform is Thomas H. P. Gould, associate professor of mass communications at Kansas State University in Manhattan, KS. The title of his book—Do We Still Need Peer Review?—contravenes Ian Betteridge’s law of headlines, which states that any headline ending with a question mark can be answered “no.” Gould answers his question with an equivocal “yes, however …” His book is not a scourge of peer review, nor is it a partisan defense. Rather, the book is Gould’s attempt to reform peer review in order to save it—and us—from “the brink, the utter doom, that is ahead.” (p. 2)
What is this doom? It consists of peer review’s well-documented failings, the advent of digital publishing, and especially what Gould terms “the rise of the Individual,” specifically his or her ability to bypass peer review and publish regardless of quality. (p. 5) As Gould sees it, search engines trawl an ever-expanding ocean of detritus, giving us a “simple, easy, largely useless way to gather research of the most minimal value in a very efficient fashion.” (p. 83) He worries about the greenhorn researchers and graduate students who, pressured to publish and contemptuous of a flawed peer review process, ultimately “feel free to publish without peer review at all.” (p. 86)
As doomsday scenarios go, this is rousing stuff. The problem is that the evidence does not support it, and the same is true for much of this frustrating and incoherent book. The aforementioned graduate students are just a few members in Gould’s army of straw men. Time and again we are told of the “some” and the “many” and of their strong feelings about the topics under discussion. (p. 1, 11, 61, 63, 71, 79) But for evidence Gould submits only his hunches. In one particularly embarrassing example, Gould sets up a straw man, denies that he’s done so (“This is suggested not as a ‘straw man’”), and then proceeds to knock the stuffing from his absent enemy. (p. 60) [End Page 197]
When evidence is adduced, it is skimpy, or worse, misinterpreted. For example, Gould invokes four studies of the Deep Web, the huge volume of content that conventional search engines cannot access, to demonstrate that “graduate students are highly prone to use only the top level of the Web,” but only one paper makes that claim, and in its literature review no less. (p. 84) In another, more glaring example, Gould misidentifies institutional repositories as e-reserves, a mistake that continues throughout the book. (p. 81) He also misstates library intentions for them and then claims that he “poured through a multitude of books and research articles” but could find no “chatter about this thing we refer to as ‘publishing’ in the e-reserve era.” (p. 82)
After a blunder of this size, what reader can be expected to continue, much less trust in the author’s analysis? And I have not mentioned the many digressions, dubious claims, and syntactical errors that precede this jaw-dropping statement. For all his concern about blog researchers, readers interested in the reform of peer review will learn more from Cameron Neylon’s posts than from this slipshod monograph.
Writing for Nature in 1977, the biologist Thomas Jukes quipped, “Publishing a book is a way of avoiding peer review.” Failing the surprise announcement of a sting—such as the phony, error-ridden cancer study accepted by...