Looking back on four years of Published Peer Review at PLOS

Written by Lindsay Morton

Over 4 years: 74k+ eligible articles. Nearly 85k signed reviews. More than 30k published peer review history packages, almost 15k of them with at least one signed review.

A little over four years ago, PLOS announced Published Peer Review History, which offers accepted authors the option to have their peer review history—including editorial decision letters, with peer reviews, and the authors’ own responses to reviewer feedback—published alongside their article. If a reviewer chooses to sign their comments, their name also appears publicly. Our approach to published peer review is an example of how we work to embed Open Science directly into the day-to-day publishing program, making research transparency a natural part of researchers’ experience, while also respecting community norms and individual needs.

Since its introduction, about 40% of PLOS authors have opted-in each year, with a slight increase over time. About 20% of articles include both Published Peer Review History and one or more signed reviews.*

Author opt-in by journal and discipline

In our 2022 update we presented data for the seven established PLOS journals (PLOS Biology, PLOS Medicine, PLOS Computational Biology, PLOS Genetics, PLOS Pathogens, PLOS NTDs, and PLOS ONE). Now we are pleased to include numbers for the new journals founded in 2021 as well (PLOS Climate, PLOS Water, and PLOS Sustainability and Transformation).

In line with previously observed patterns, our highly selective journals, PLOS Biology and PLOS Medicine, tend to have the highest rates of author opt-in. Select journals in both the biology (PLOS Computational Biology, PLOS Genetics) and health sciences (PLOS Global Public Health and PLOS Digital Health) fields also exhibited relatively high rates of opt in at or above 50%. The remaining titles hovered slightly below 40%.

Examining PLOS ONE’s author opt-in rate by topic area section shows a willingness to share Peer Review History among authors in medical and adjacent fields, perhaps because signed and published peer review is more common and therefore more normalized in the medical community.**

Subsection	Opt-in rate
Women’s health	49%
Urology	49%
Critical care and emergency medicine	48%
Gender and sexuality studies	48%
Pulmonology	48%
Obstetrics and gynecology	48%
Clinical trials	47%
Neurology	46%
Nephrology	46%
Computational biology	46%
Surgery	46%
Health care	45%
Pediatrics	45%

Author opt-in by region

When viewed by geographic region across the portfolio, opt-in rates are highest among authors affiliated with institutions in Africa (excluding the North African countries tallied as part of Middle Eastern and North Africa, or MENA), followed by Europe, Australasia, and the Americas.***

Reviewer signing

The rate of reviewer signing has gradually increased from 15% to 21% since we began tracking this data in a reportable way, also in May 2019.

Reviewer signing by journal

The percentage of individual reviewers who signed their comments varies by journal. Similar to the pattern observed in author opt-in rates, health sciences titles, including PLOS Global Public Health, PLOS Medicine, and PLOS Digital Health tended to have higher rates of reviewing signing, while biological titles like PLOS Pathogens, PLOS Genetics, and PLOS Biology, and PLOS Computational Biology were lower. Journal selectivity and editorial model do not appear to influence reviewer signing rates in the same manner that they impact author opt-in—perhaps because reviewers have less contact with journal staff editors.

Reviewer signing by decision type

Across the portfolio, the likelihood of reviewers signing their comments corresponds with their decision recommendation. Reviewers are most likely to sign when recommending an accept or minor revision decision, with the likelihood decreasing as recommendations become more critical. 

This remains true whether or not the manuscript ultimately goes on to become a published research article, however it does mean that published articles are more likely to have at least one signed review than manuscripts that receive a reject decision.****

Looking ahead

Over the past four years, PLOS has published more than 30,308 research articles with Peer Review History. When introducing the option four years ago, our initial expectation was that the practice would normalize over time and that we would see an increasing proportion of authors elect to make their Peer Review History open. This has generally not been the case. 

PLOS authors have chosen Published Peer Review History at a fairly consistent rate of about 40% (rising gently from 38% in 2019 to 42% in 2023). Despite the benefits inherent in a more open and transparent scholarly communication system, the stable rate of uptake suggests that concerns also persist. While we haven’t encountered the realization of these concerns in our experience so far, the evidence base is still poor. As Ross-Hellauer, Bouter and Horbach have expressed recently, “to responsibly open up peer review, there is an urgent need for additional evidence.” We hope that the significant dataset of Public Peer Review Histories now available from PLOS can help support a research agenda in this important area.

Notes

*Dataset covers manuscripts submitted on or after May 22 2019, when Published Peer Review History was introduced, and includes all activity through December 13 2023, when data was collected. Published Peer Review History is available only for articles submitted after May 22 2019.

**Includes only topic sections with a minimum of 100 publications in the dataset.

***In this dataset, region corresponds to the primary affiliation of the corresponding author. Each article is counted only once, even if the authors are affiliated with multiple institutions in more than one region.

****Since manuscripts may be reviewed more than once, totals may add up to more than 100%.