On March 31 2021, PLOS Computational Biology introduced a new journal requirement: mandated code sharing. If the research process included the creation…
It’s Peer Review Week! This year’s theme addresses “Transparency in Review.” Transparency can mean a lot of things–for some, it entails a fully opened review process with signed reviews made available after publication; for others it might mean public comments on public preprints before formal peer review; and for others, it might mean something entirely different! In this post, I’m going to speak from my perspective as a volunteer academic editor for various open access journals (approaching something like eight years experience, variously for PLOS ONE and PeerJ). Note that this is personal opinion, not necessarily explicit policy of any organization with which I am associated.
Specifically, I want to look at how reviewers and authors can help ensure a transparent peer review process. I’ve solicited, read, and interpreted reviews and rebuttals from a variety of people over the years, for a variety of papers covering the spectrum of topics and initial manuscript quality. Based on this experience, I would suggest the following…
- If you are given the opportunity to suggest reviewers for your paper, be honest about potential conflicts of interest. As an editor, I totally accept that some people probably aren’t well suited to review your hard-earned manuscript; it’s much easier for me as editor if you can give me a specific reason why they should be excluded. I don’t need a paragraph of gossip, but a simple “We have had prior deep-seated disagreements with this researcher on previous manuscript reviews” can be quite helpful. It doesn’t mean I’ll necessarily believe you (especially if you give a string of 20 people to exclude), but usually I give the authors benefit of doubt.
- Make sure all necessary supporting data are available for the editor and reviewers. If you are doing a phylogenetic analysis, don’t just give me a paragraph with the codings for the three characters you scored for that taxon. Provide the full TNT or NEXUS file! Believe me…it annoys everyone to have to reconstitute your matrix, and the possibility of errors are quite high. Even if you aren’t required to make data public, do it anyway (if permitted).
- Make sure that your paper abides by all relevant ethical guidelines for your field. Some authors seem to think that if you submit a paper to a non-society journal, ethical guidelines magically don’t apply. Sometimes this gambit “works”, especially if the handling editor isn’t completely familiar with relevant expectations, or if the authors obfuscate the ownership of a specimen that’s outside of a permanent museum collection. Don’t do this.
- When responding to reviews, address each point individually, especially for issues related to scientific content. Sure, it’s okay to say “we incorporated all minor stylistic and grammatical suggestions,” but don’t tell me “We followed all of the reviewers suggestions for revision of the phylogenetic analysis,” because that is almost never completely true. Give a point-by-point summary. As editor, I’m going to read through the reviewers’ comments, and I’ll be really annoyed if you miss something obvious. Worse yet, I’ve sometimes had suspicions that authors tried to pull a fast one on me, by downplaying a valid major criticism from the reviewers. There lies the path to rejection, or at the very least a request to revise again. It’s a waste of my time, and a waste of yours, so get it right from the start!
- Make your pre-review manuscript public and solicit feedback! I did this with a recent paper, and was very pleased with the results. Not only is it transparent, but it greatly improves the potential end product. Advertise your preprint’s availability on social media, and take the feedback seriously. Make the revised preprint public, too!
- When solicited for a review, be honest about any conflicts of interest. If you co-authored with one of the authors before, let the editor know. If you’ve had a personal or professional conflict and feel that it would affect your review, let the editor know. They may still ask you to do the review (especially if it’s a small subfield; there may not be many choices)–and you can then do so with a clean conscience, and the editor can read your critique with fully open eyes.
- Read the reviewer guidelines, and don’t agree to anything unless you’re certain you can and will abide by them. This especially applies to cases where reviews might be made public!
- Stick to deadlines as best you can. If you can’t make a deadline, let the editor know. We’re generally pretty forgiving on this; life happens sometimes. But don’t do the “ghost reviewer” thing and ignore days and weeks and months of reminders. It holds up the process, and isn’t considerate of your colleagues.
- Sign your review (if appropriate). Although there are certainly cases where signed reviews might be inappropriate, my personal preference is to reveal my identity to the authors. Your mileage may vary.
- Unless explicitly requested by the journal, don’t make accept/reject/revise recommendations in your comments to the authors. This is a decision for the editor, and is based on journal-specific standards.
- If you spot a serious ethical issue, let the editor know. It may or may not be appropriate to flag it in your review that is visible to the authors (especially if you are not completely certain of a particular issue), but in any case let the editor know.
- Be appropriately specific and transparent. Editors and authors aren’t mind readers…give specific citations that provide evidence for critical advice, especially if you are asking for a new type of analysis or if you posit that the authors did something incorrect.
- If you have an in-press paper that might help out the authors, consider mentioning it to them so that they can include it.
- Accept that there is more than one way to skin a cat, and there is more than one way to write a paper. Be specific about which comments are absolutely necessary for the scientific integrity of a paper, which would be helpful but optional, and which are strictly personal preference. Is that expensive and time-consuming extra work you propose essential, or could it be addressed by adding a few caveats? We all want to do the best science–but be wary of wielding scientific rigor as a sledgehammer.
I could of course pontificate on this topic for ages…but will cut my commentary off here. Drop a note in the comments if you have thoughts, opinions, or other commentary on transparency in peer review!