"Peer review" is younger than you think. Does that mean it can go away?

This is a blog post I've had sitting around in some form for a few years; I wanted to post it today because:

1) It's about peer review, and it's peer review week! I just read this nice piece by Ken Wissoker in its defense.
2) There's a conference on argumentation in Digital History this weekend at George Mason which I couldn't attend for family reasons but wanted to resonate with at a distance. 

It's still sketchy in places, but I'm putting it up as a provocation to think (and to tell me) more about the history of peer review, and how fundamentally malleable scholarly norms are, rather than as a completed historical essay in its own right. [Edit--for a longer and better-informed version of many of these points, particularly as they relate to the sciences, Konrad Lawson points out this essay by Aileen Fyfe; my old grad school colleague Melinda Baldwin has an essay in Physics Today from her forthcoming project that covers the whole shebang as well, with a particular emphasis on physics.]

It's easy, when writing about "the digital," to become foolishly besotted by the radical transformation it offers. There's sometimes a millenarian strand in the digital humanities that can be dangerous, foolish, or both, and which critics of the field occasionally seize on as evidence of its perfidy. But it's just as great a betrayal of historical thinking to essentialize the recent past as to hope that technology lets us uproot the past. We should not fall short of imagining the changes that are possible in the disciplines; and we shouldn't think that disciplines need revolve around particular ways of reviewing, arguing, or producing scholarship.

Here's a short historical story about one thing we tend to essentialize, peer review. I find it useful for illustrating two things. The first is that scholarly concepts we think of as central to the field are often far more recent than we think. This is, I think, a hopeful story; it means the window for change may also be greater than we think. The second is that they are, indeed, intricately tied up with social and technological changes in living memory; the humanities are not some wonderful time container of practices back to Erasmus or even Matthew Arnold. I'm posting it now, after delivering it as a hand-wavy talk at Northeastern in 2015. 

Peer review seems to be so fundamental to scholarship that we can hardly imagine a world without it. Conventional histories of peer review suggest that it is old indeed. Kathleen Fitzpatrick starts her discussion of its history in the 1750s, although she suggests that the "history of peer review thus appears to have been both longer and shorter than we may realize," extending back to the 17th century but still imperfect by the 1940s. Wikipedia editors are more firm in their straightforward assertion that is was developed by Henry Oldenburg (1619–1677), who built on the work of Ishāq ibn ʻAlī al-Ruhāwī (854–931). (Wikipedia is less clear on just how it quietly gestated for 7 centuries.)   

But even if peer review is ancient, "peer review" itself is quite new. I was surprised, a few years ago, in performing anachronism consulting for the show "Masters of Sex," set in the early 1960s, to see my algorithms reject one character's suggestion that Masters and Johnson needed to publish in peer reviewed journals as hopelessly anachronistic. But that is indeed the case. Google Ngrams shows only sporadic uses before about 1970; the adjectival form "peer reviewed," as adhering to scholarship, barely exists before 1980. (As always, you should basically ignore Google Ngrams results from after 2000, but why not include them?)

Of course the thing may exist before the word: but one thing I've found invariably in looking at these etymologies is words usually do not march straight out of the primordial ooze into widespread use for no reason at all, particularly words describing so specific and unpoetic as a practice like this. Usually there is some reason, some new thing in the world that requires a new term to distinguish it from what has come before.

So what new thing gave rise to peer review? Reading through the texts gives some sense. JStor contains no uses of the phrase until 1965, in reference to “peer review groups” at the NIH. Through the late 1960s the phrase was only used in the context of doctors supervising medical care of other doctors. The first use outside of medicine I find is in 1969, in a library sciences context,1 also about professional self-evaluation. An early usage for grants is in 1970.2 The first usage of the phrase in the American Historical Review is in 1978, in a decidedly negative evaluation of the “peer review bureaucracies of foundations and government,” “manned by scientists of lesser achievement.”

So what is the new thing being described here? As best as I can tell, the reason for its existence is the rise of the new government funding bureaucracy in the 1960s; the NIH, the NSF, and their smaller cousins like the NEH all needed mechanisms to distribute their new government largess: and peer review was a way to ensure that government money was not handed out by the government, but by experts from the scientific community. (This story, by the way, bears an interesting relationship to the one I wrote earlier this month about government attributes versus public ones; that sentence would have a different valence if I said that scholars wished to ensure the "public money was not handed out by the public.") Its use in non-publication situations--peer review boards investigating medical malpractice, for example--is almost entirely about protecting professional organizations from state or other bureaucratic interference. And when there are large-scale discussions of peer review in science--as in a special 1985 edition of "Science, Technology, and Human values," they frequently take grant-making as the archetypal form, not the refereeing of scholarship.

The central technology that causes this new term to spread to scholarship, then, is the grantmaking state. But this relies on a more prosaic technology, as well. Peer review is the archetypal form of scholarly organization in the age of the xerox machine. Without it, a sheaf of grant proposal could not be easily snapped into a binder, mailed across the country, and then (perhaps) flown back to Washington with its expert reader for discussion.

Digging around a little recently, I see that historian of science Alex Czsisar wrote a short piece for Nature in 2016 (after I gave this as a talk, so not fully incorporated here) where he says this, which is very much along the same lines.

'Peer review' was a term borrowed from the procedures that government agencies used to decide who would receive financial support for scientific and medical research. When 'referee systems' turned into 'peer review', the process became a mighty public symbol of the claim that these powerful and expensive investigators of the natural world had procedures for regulating themselves and for producing consensus, even though some observers quietly wondered whether scientific referees were up to this grand calling.

All of this suggests, though it doesn't prove, that the shift to a language of "peer review" involves a model of research that draws on a nationally organized scientific funding system that merges with a series of older traditions. Most of the histories of peer review in the sciences note how late journals were to adopt it: leading British publications like the Lancet and Nature don't take up outside peer reviewers until the 1970s.

If the history of peer review in the sciences is young, the history of peer review in the humanities is even younger. The earliest usage in the front-or-back matter of the American Historical Review that I see at first glance is from 1996, in describing who can perform book reviews; the second use outside a history-of-science context is in 1997, when Sheila Fitzpatrick notes unhappily that a book has obviously "undergone extensive peer review" in a way that weakens it, serving as an "uncomfortable reminder that peer review may function not only as a gate-keeping procedure but also as a kind of censorship of unpopular opinions." (Here's a similar 2012 argument in favor of editorial review, *not* peer review, in a science journal). (Not far down the list is Ayers and Thomas's 2003 digital article "The Differences Slavery Made;" rather than showing up to challenge decades of consensus on peer review, digital scholarship was already arriving on the hardly after the consensus had set).

I'd have to do more research to really understand what the outside review policies of leading journals were in the (say) 1950s and 1960s. I have the impression, but have lost the reference, that at least single-copy referee reviewing was common, if not mandatory, in the period. ("Outside referee," though, is another 1960s neologism.) Still, the technology of mimeographs and carbon copies makes for different forms of outside review: In 1956, the AHR only demanded a single paper copy of each article submitted. (The ribbon copy, please; keep the carbon copy for yourself.) This seems to have been still true in 1970; I suppose by then the AHR could have been paying for photocopies on their own, but I doubt they did. By 1980, the number of required copies number has increased to 2; by the late 1990s, the frontmatter demanded four, or a Microsoft-compatible disk, which is what it would take for a really modern peer review system. But surely I've said enough now that someone will chime in in the comments about how mimeographs were sent to various reviewers in serial.

I know even less about monographs; but it should be widely known that some important work in the field *continues* to be published as part of edited volumes, trade presses, and other channels subject only to editorial review, not peer review.

So what? Obviously peer review didn't *really* come into being in the historical profession in 1995. That's ridiculous! Books and journals had various forms of outside referees in the mimeograph/carbon copy age as well. But the point is this: something did change, and more recently than we might think. Peer review was created, rather than always existing in the humanities; its rhetorical adoption in those fields as a core term may be less connected to eternal ideas of scholarship, and have more to do with the effort to make them more like the sciences in the postwar university. Be skeptical of the administrative-ese, buzzword-inflected vogue for the digital humanities all you want: but also, be aware that if you say "peer review is fundamental to sound scholarship," you're speaking in the buzzwords of not long ago, yourself.

I wouldn't argue in good faith that peer review is new, so it's bad. It might be that the scholarly practices from 1970 to 2010 are indeed worth preserving.

But at the same time, I'm not sure that digital scholarship could ever fully reconcile itself to peer review in the traditional sense without transforming it enough that we'll need a whole new phrase to describe the new regime to come.

Randomly selected bibliography:

Some early citations, and some previous histories.

“Atlantic City Conference: A Great Show–in Two Parts and a Cast of Thousands.” ALA Bulletin 63, no. 7 (): 915–964. http://www.jstor.org/stable/ 25698237.

Cooper, Joseph D. “Onward the Management of Science: The Wooldridge Report.” Science 148, no. 3676. New Series (): 1433–1439. http://www.jstor.org/ stable/1716537. 


Csiszar, Alex. “Peer Review: Troubled from the Start.” Nature News 532, no. 7599 (April 21, 2016): 306. doi:10.1038/532306a.