Note: The National Institutes of Health (NIH) recently requested feedback on its planned implementation of the White House Office of Science and…
Who’s to blame (or credit) for fast peer review?
Lately around the blogosphere and Twitterverse, I’ve been seeing an increasing number of folks complimenting open access journals on their quick turnaround for peer review–or blaming open access journals for slow review. Fast turnaround is an excellent thing, no doubt, and slow turnaround is a bad thing. However, the implication of some of these statements is that open access publishers are doing things quicker (or more slowly) than non-open journals, simply by the virtue of being open access (or being PLOS or PeerJ or choose your favorite publisher). I’m not convinced of this, and here’s why.
Peer review is done by peers. Not by journals, not by content management systems, not by the paid editorial staff at the journals, not by the numerous volunteer editors. Based on my experiences as a volunteer editor at two different open access journals (full disclosure: PLOS ONE and PeerJ), author on papers submitted for numerous other journals, and conversations with colleagues, at least 80% of a speedy turnaround rests with the reviewers.
If a reviewer finishes his or her review in a timely fashion, you’re probably going to get your paper back in a timely fashion. If the reviewer drags his or her feet for six weeks, you’re going to have a slow time of it. Now, the journals and editors can help things along, by nagging tardy reviewers (I’m getting better at this with more experience). Journals can also set short but reasonable deadlines (perhaps 2 weeks rather than 4 weeks, for instance). But at the end of the day, it’s up to the reviewer to be on time.
Our implicit standard for what makes a “good” reviewer rests on their fairness, expertise, thoroughness, and timeliness. Timeliness in turn is dependent on a few other factors–personal schedules and manuscript length in particular. It’s not terribly reasonable to expect a reviewer to submit their review in 10 days if those days span Christmas and New Year’s. Neither would I expect a 10 day turn-around for a 150 page manuscript (some reviewers have surprised me, though!). On the other hand, if a reviewer accepts an invitation, they should be willing to do the work on deadline (or pretty darned close to it). It all comes down to the “Golden Rule.” I’m annoyed when my paper sits in reviewer limbo for three months; why should I subject someone else to these sorts of delays, if I can help it?
So, are open access journals any better or worse than non-open journals? This is an inherently testable claim (particularly when journals post relevant data), but for now I’m going to speculate. For one, I think it depends a lot on the paper itself (particularly length). For another, reviewers may be more excited about reviewing particular papers or for particular journals. Maybe some reviewers are quicker for open access journals. Note that in all cases the responsibility is largely with the reviewer! Journals can help this along, though, by selecting reviewers known to do a good job in minimal time, and avoiding reviewers who chronically delay things (note that we all have our slow times, though!). But, the reviewers rule in this system. Timeliness depends on them. Let’s not forget it.*
*Some might argue that this is a strike against pre-publication peer review. I’ll freely admit that slow reviewers are a problem, but I’m on record elsewhere in favor of pre-publication peer review, coupled with liberal use of preprint servers, is a good thing. I trust my own work more after it’s been reviewed, and hold others to the same standard.
I agree, although it varies- I had a paper languish with PNAS editorial board for a month, then senior editors considered it for another month, before it went for peer review. And I’ve heard plenty of horror stories of editors that just forget about papers (rare, perhaps, in this digital publishing age) or (more commonly) don’t hassle reviewers. But most editors are fast, because our job is simple (unless we’re the rare type that actually does all the proofreading and word-by-word checking and detailed editing– hopefully getting paid for what would be a full time job!). Reviewers do the hard work, and a good/bad reviewer makes all the difference. Fundamental rule should be: if you are asked to review and can’t 100% certainly do the review in the time the journal asks you to do it, don’t accept the invite! So simple, but as an editor I’ve so often seen this blithely ignored, which comes across as disrespect for the authors and the process.
Another problem that affects turn-around time is getting reviewers. You can’t really ask 10 to review if you only want 2, so you’re stuck waiting for people to agree or decline before asking the next person to review. If lots of people decline, you’re not going to have a short review time.
Well, Andy, obviously you’re right that a review cycle can’t be completed more quickly than the reviewers do their jobs. And equally obviously, the mere choice of licence a journal uses doesn’t directly affect turnaround time.
And yet, my intuition (based on anecdote rather than any systematic data) is that open-access journals, and particularly the “new approach” journals, do pretty consistently get papers through review more quickly than traditional journals.
Why is this? My best guess would be just that the new journals tend to be inhabited largely by young, enthusiastic workers who’ve not had time to ossify and become acclimatised to the expectation that everything takes forever — that’s especially true of people who, like yourself, are young enough to have grown up with the Internet; and even more true of the visionary journals that are on a mission, and tend to attract editors who are sympathetic to that mission. By contrast, the editorial boards of traditional journals tend to be manned by people who grew up in a very slow-moving academic world, where a year in review and another in press was just normal. (Of course, there are plenty of exceptions in both directions — I am describing what I perceive as the general trend.)
If that’s so, then the quicker turnaround speed of new journals is a consequence primarily of expectation — of a culture what getting things done quickly is standard. And because many more new journals are being started as open access than there are old journals converting, anything that’s an advantage for new journals also manifests (by correlation, not causation) as an advantage for open-access journals.
You might be interested to know that at we’re launching COPE’s (Committee on Publication Ethics) new Ethical Guidelines for Peer Reviewers at tomorrow’s European Seminar in London http://publicationethics.org/cope-european-seminar-2013 (twitter #c0pe; NB 2nd place is a zero, not oh).
I know from experience that many researchers don’t get guidance on any aspect of being a peer reviewer – what it entails, how to review – and so may be unaware what their ethical obligations are. The COPE guidelines give the basic principles and then go on to list expectations during the peer-review process – what do on being approached to review, during review, when preparing the report – and expectations post-review.
Hopefully, the guidelines will be helpful to researchers at all career stages (and to institutions in training their researchers), but especially to young researchers. Not only as general guidance, but also to use as a reference they can point to when they find themselves in difficult situations, e.g. when their PI keeps asking them to review manuscripts for them but the journals never get asked/told and the young researcher never gets credit for those reviews. This also deprives them of the chance of getting known by those journals and of building up a body of reviewing with them and the benefits that go with that.
Exactly! Recognizing that reviewers sometimes have circumstances pop up that mean they can’t do a timely review, it’s also helpful to let the editor know if this is the case. I far prefer that to waiting a week or two weeks beyond the deadline, or having to chase down the reviewer myself.
In my own experience, I’ve found _most_ reviewers are quick to respond with an accept or decline. However, if I haven’t heard back from them in two or three days, they’re generally pretty much a writeoff. Thankfully, most that I’ve dealt with are pretty prompt, at least in paleontology.
This seems like something that’s ripe for testing! I can say, anecdotally, that things are much faster now that online publishing is the norm (a few years back, I once waited six or eight months for an MS!). . .whether this is due to OA or not, I’m not certain. Different journals definitely have different cultures in my experience–i.e., I’ve found high variance in my dealings with OA journals. In general it seems like smaller editorial boards mean slower turnaround, if only because a lone editor may be more overworked. No matter what the cause, I welcome the increased overall pace!
In any case, as you note and I mentioned above, perhaps OA does help hurry some people along! I’m not sure if it’s something I consciously think about while doing a review, but others may have different experiences. I would never consider taking more than a month to do a review, for any journal (recognizing this was not past practice for many). Any thoughts from others reading this? Do you review more slowly for some journals than others?
Thanks! That promises to be quite useful…I will confess I basically learned by example, and a more concrete set of guidelines would have been helpful.
The whole idea of a PI making his or her trainees do reviews is completely alien to me–my sense is that this is virtually never done in paleontology. It’s probably directly related to the way our graduate training is run–students tend to be much more self-sufficient than in, say, physics or chemistry or the biomedical sciences.
I often come across this – it’s one of the most common things I get asked about in workshops – and it can involve a large number of manuscripts. One researcher recently told me she’d done more than 20 reviews for her PI without acknowledgement and was desperate to find out how to deal with this.
[…] Who's to blame (or credit) for fast peer review? | The Integrative … Tags: a-newly-minted, comparative, global-health, health, integrative, medicine, merolog.com, […]
We’ve been working on creating a standardized, portable peer review that goes through three reviewers and can be returned to the author in 1-2 weeks. One thing we have noticed is that the quality of the match between the paper and reviewer is very important to the speed of the review. Reviewers who are genuinely interested and qualified to give feedback on a specific paper are very likely to agree to the review and return it quickly. Our ideal model (once we have a very large network of reviewers) is to create more of a “pull” system, where we match the paper to a group of experts on that topic and provide the platform for the reviewers to claim the paper if they are available and the paper is of interest.
[…] Who’s to blame (or credit) for fast peer review? | The Integrative Paleontologists Lately around the blogosphere and Twitterverse, I’ve been seeing an increasing number of folks complimenting open access journals on their quick turnaround for peer review-or blaming open access journals for slow review. Fast turnaround is an excellent thing, no doubt, and slow turnaround is a bad thing. Share […]