In Defense of Law Reviews

As I get ready to take the plunge into graduate-level political theory, I've had to start thinking a little more about peer review. Peer review is, of course, what most academics consider to be the sin qua non of academic publishing; if it is not peer reviewed, then it might be utter gibberish for all we know. Law professors have a bit of an inferiority complex on this issue, as we have a unique publication process that does not include peer review. And peer review isn't the only difference between the disciplines: law articles are notoriously lengthy, notoriously footnoted, and notoriously go through a frenetic multiple-submission/expedite process to find their eventual home.

Recently I've been getting a little more exposure to the peer review process -- through a refereed conference (and follow-up journal special issue) and by developing contacts with professors in other disciplines who, just like law professors, love to gripe about the shortfalls and pitfalls of their own publication process. I've also had the opportunity to read more peer-reviewed articles -- primarily in political theory, public philosophy, and psychology -- and compare them to what I typically find in a law review. And the more I think about it, the more I think the law review system -- which is viewed with incredulity by essentially every non-law professor I've ever described it to (and by many law folks too) has a lot to say for itself.

Let's list what I take to be the key distinguishing characteristics between legal academic publishing versus the general norms:

• Most law articles are selected by second- and third-year law students on various law journals. They are usually not blinded. My understanding is that sometimes upper-level graduate students can serve as peer-reviewers, but mostly the reviewers are other professors in at least a related field studying a blinded manuscript. 
• Law reviews are much longer than peer reviewed articles. A "short" law review piece (~20,000 words) would be considered quite long in most other disciplines. Law review articles also have a tremendous amount of footnotes.
• Law articles are typically accepted or rejected "as is". Rejections come without any comment or feedback. Peer review offers a continuum of responses (accept, conditional accept, revise and resubmit, reject), and authors receive the reviewer comments.
• It is customary for draft law articles to circulate on sites like SSRN even before they're submitted for publication. This is a shakier proposition in other disciplines, as some peer-reviewed journals will outright refuse to consider a piece that has been "published" on such an online repository.
• Other disciplines submit articles to one journal at a time; moving on to the next one only after a piece is finally rejected at the journal it's currently in front of. Law review articles are submitted to multiple (often upwards of one hundred) journals at once. They move through the selection process via "expediting" -- the initial offer to publish (and deadline for decision) is forwarded to all the higher ranked journals, who then race to make their own decision. If a higher-ranked journal decides to make an offer, the process continues. Repeat until the deadline expires or no higher-ranked journal remains to expedite.
• Related to the above, law articles have a much faster turnaround from submission to acceptance.

So which of these is better? In my view, the goal of an effective publication process should be (a) to produce good articles, both in terms of making novel and effective contributions to their field and in terms of being informative and helpful for other members of the academy, and (b) to place articles roughly according to their merit -- that is, the best articles should be in the top journals, the pretty good articles should be in a pretty good journal, and so on.

On these fronts, I think law reviews perform alright. In terms of overall quality, I'd first observe that I see little difference in the strength of the articles published in the top half or so law reviews versus those published in a typical peer-reviewed context. The fact that there are so many law journals does mean that one starts to see some real iffy contributions towards the bottom of the chain (though to be fair, I've seen some utter dreck in the peer-reviewed world too). But the offerings of reputable law journals are I think every bit as strong as those of equivalent peer reviewed outlets.

I also think the internal distribution of articles is more or less on point. An article published in Harvard or Yale or California or Chicago strikes me as being as likely to make a big interdisciplinary splash as those in the flagship journals of other disciplines. And outside of that, I feel like most articles I read feel roughly rightly-placed vis-a-vis their merits. I know with respect to my own work that I've generally been pretty content with where my pieces have ended up -- sometimes I think an article underplaced a little, sometimes overplaced a little, but never wildly off-kilter with how I perceived its overall merit.

That law reviews actually do a pretty decent job selecting for quality may be surprising, given the lack of peer review. People often comment on how terrible it must be for untrained law students to hold so much sway. But the thing is most people undergoing peer review don't seem to have a ton of confidence in peer reviewers either. They're replete with horror stories about articles badly butchered or barely read at all. Basically, it sounds to me like most folks undergoing peer review are repeatedly rolling the dice until they get a reviewer who is generally sympathetic to their aims and committed to giving their piece due consideration.

Of course, that broadly describes law reviews too -- we all know that many of our submissions are barely skimmed or are rejected for reasons that, if articulated, would make us want to scream. In other words, both peer review and law review submissions require two things: the piece has to be up-to-snuff quality wise for the selected venue, and then you also have to get lucky and have it be read by the right type of reviewer.

And this is where law reviews' multiple submissions scheme is really good. As much as we like to agonize over it, I think it's fair to say that journals fall into broad "buckets" of prestige within which they're relatively interchangeable. There are exceptionally good journals (Yale, Harvard, Chicago, California, etc.), very good journals (Minnesota, BU, BC, GW, Notre Dame, etc.), quite good journals (Arizona, Florida, George Mason, Georgia, etc.), and so on. I imagine the same is true in other disciplines. And most of us, if we're being honest, have a decent idea of where our articles deserve to be in that pantheon. Now, let's say that in each "bucket" there is one journal that has that lucky confluence of a good reviewer who will give your piece its due. If you're submitting sequentially, it might take up to ten tries before you hit that jackpot (and that assumes you haven't lost faith about how good your piece is). But with multiple submissions, it doesn't matter if nine of the ten relevant journals reject your piece without reading it; you only need the one offer. Placing the piece in front of every journal at once means you get to pull the slot lever of every machine. The multiple submission process may be frenetic, and it certainly isn't perfect, but I believe it does a faster and more efficient job of matching articles to their proper academic homes.

Finally, I'll forward a proposition that I think will be quite controversial but I'm willing to defend: the structure of legal publishing produces better articles. The jabs against law review articles are that they're long and excessively footnoted. The former doesn't bother me at all; I just view legal articles as replacements for the books we don't write. The latter is probably a sop to the student editors who are not, of course, experts in the field. But that's a good thing too: one great thing about legal scholarship is that it is generally relatively accessible to novices. If I'm trying to learn about a new area of law, I can pick up pretty much any article in the field and will be greeted with a buffet of relevant sources and explanations that make it easy to quickly pick up the basics of what they're talking about. This is very different from other disciplines, where if you don't come in with a pretty healthy grounding to start you'll be utterly lost within four pages. I think this is attributable to the fact that while other academics write for one audience -- fellow specialists in their subfield -- legal academics write for two. We do write for a subcommunity of specialists, but we also write for the intellectually curious amateur, a/k/a, the law review editor. And I think keeping that second audience in mind produces scholarship that is more lucid, more practical, more cogent, and ultimately more meritorious than that which is found elsewhere.

Now, none of this is to say that peer review lacks for benefits. But some of these (like blind review of manuscripts) could be easily incorporated into the law review process. And others (like reviewer comments) probably cannot be, but are easily replaced by the norm of pre-publication distribution of SSRN drafts and other like ways of soliciting feedback on papers before, during, and after the publication process.

Law professors often lack doctorates, and so we sometimes feel like step-children to the "real" academy. But I think we should buck up. I think the state of legal scholarship is actually quite strong. It produces quality articles, and sorts those articles in rough accord with their merit, and it channels scholarly writing in a direction that I think is more accessible and effective than that which prevails elsewhere in the academy. Our bizarre, idiosyncratic, unique system of publication is actually something we can be quite proud of.