The Opposing Thumb

An opinionated digit leafs through the biological literature
Media Roundup

The Spherical Frictionless Horse

This week Nature has made a bit of a splash with the news that it is experimenting with a “double-blind” review format. The standard review process for a scientific manuscript is “single-blind”, i.e., the referees evaluating the manuscript know the identity of the submitter(s), but the authors don’t know who the referees are (typically between two and three referees per manuscript, with the third referee one of the prime bogeymen of modern science, “eat your vegetables or the third referee will demand 8 months of additional experiments and then reject your paper anyway). The standard justification for the standard review process is that the standard human being is a vindictive SOB. This is not an entirely erroneous view of human nature, and there is a pervasive fear, particularly among younger scientists, of upsetting one or another senior authority in their field. Understanding this requires the final piece of the process: manuscripts are reviewed by working scientists, on top of doing research, teaching, and (the larger chunk of modern scientist’s time) scrounging for money (also known as applying for grants). It is considered part of good scientific citizenship, and it is normally done for free. Let the record also show that reviewing manuscripts is a lot of work.

“Double-blind” reviewing is an admirable attempt to counteract other pervasive aspects of Homo not so sapiens: on the one hand, the tendency to be unduly impressed by celebrity or flashy credentials; on the other, the prevalence of prejudice against newcomers, particularly those outside the more ‘prestigious’ scientific centers. The gist of the argument is simple: a scientific manuscript should be evaluated on its merits alone- regardless of the author’s reputation and/or the IvyLeagueness/Oxbridgidity of the institution where the work is done. Let’s be clear here: many reputations are well, and hard, earned. But the plain fact is that arguments from authority should not count for anything in the evaluation of a scientific claim. Hence, “double blind” reviewing. The referees don’t know the identity of the research group, any more than the authors know the identity of the referees.

Double-blind is how the process should function. The only exceptions, in my opinion, are those cases where the author (and here, in the multi-author reality of modern biomedicine, the one bad apple rule should apply) has a history of fraud or misconduct. If this is the sort of thing you’ve pulled in the past, that should affect how we evaluate your more recent work.

The problem is, in practice, this state of grace is next to impossible to attain. Currently, at high-impact journals, the majority of submitted manuscripts are rejected by editorial decision. That is, they are never reviewed by referees, blind or not. Rates of editorial rejection can reach or exceed 80%. Some journals, like PLOS One, have been trying to address this issue, with mixed results. Thus, to have maximum impact, the editors themselves would have to be blinded. This is of course not impossible, but it would add an extra layer to manuscript submission.

The next problem is much more vexing: the identity of the authors is often woven deep into the fabric of a manuscript. The introduction and discussion almost always rely heavily on the group’s previous work. This means that the editor’s job would change- a redacted version of the manuscript would have to be produced for referees, or perhaps even a referee manuscript with the main goals, the methods and results. Because editors attempt to find referees well versed in the field of any given submitted paper, even this will not be enough in many cases. Methods and reagents leave very clear signatures.

There are cases where double blind review may be a good option, for young researchers, or for those switching fields. For relatively unknown scientists from less glamorous locations it may be a good default option, as they have little to lose and more to gain. I’m not negative in principle about double blind reviews, and all of these objections may be proven irrelevant, or minor nuisances. But I think other trends hold more promise for leveling the playing field- in particular the various forms of post-publication peer review.

Share on FacebookShare on LinkedInShare on Google+Tweet about this on TwitterShare on Reddit

Leave a Reply

Your email address will not be published. Required fields are marked *