PLOS Publication Ethics: A frank discussion on handling difficult cases

Author: Renee Hoch, Senior Editor & Manager of the PLOS Publication Ethics Team

Since 2003, PLOS has published nearly 300,000 research articles, contributing a tremendous body of knowledge to the scientific corpus. However, our roles in scientific communications do not end at the time of publication. PLOS, like many other scholarly publishers, has a Publication Ethics team dedicated to addressing ethics and integrity concerns raised about PLOS content, many of which arise after publication. 

PLOS Publication Ethics receives a broad array of inquiries, ranging from correctable reporting errors to more serious issues, including data analysis, data integrity, or reporting concerns, study design flaws, authorship and data ownership disputes, and highly sensitive matters involving research ethics, publication ethics, and potential misconduct. Altogether, PLOS has been notified of post-publication ethics or integrity concerns for <1% of our published research articles, and thus far we have issued 380 retractions (~0.13% of PLOS research articles). We are not alone: the Retraction Watch Database¹ includes over 17,000 retracted research articles.  

Why are these issues not identified before publication

All PLOS research articles are peer-reviewed, but peer review has its limitations. A manuscript is typically evaluated by 2-4 people during pre-publication peer review, whereas after publication it is available to a much broader audience, including collaborators, other researchers, those exploring ethics and integrity in the research literature, and even the press and general public. As open access publications, PLOS articles are freely available for all to read, and each reader views the work through their unique perspective. With many more eyes on the content post-publication, it is perhaps unsurprising that issues may come to light after having slipped unnoticed through peer review.  

Furthermore, some issue types (e.g. data ownership concerns) cannot be identified from the article content alone or involve people who would not typically be involved in peer review. We do not expect these types of concern to be detectable pre-publication.

Editorial actions are not indicators of misconduct

While social media and news outlets often highlight egregious cases and/or those involving misconduct, much of what the PLOS Publication Ethics team sees may have arisen due to honest errors. Importantly, it is beyond the journal’s or publisher’s remit to adjudicate on intent or personal culpability. Regardless of who was responsible, regardless of whether issues arose due to honest errors or deliberate misconduct, and regardless of when issues are noticed, PLOS has a responsibility to take action as needed to ensure the integrity and reliability of our publications. We take this responsibility very seriously. 

What it takes to resolve publication ethics cases

Each publication ethics case – even for seemingly straightforward issues – takes its own path and involves multiple contributors. This is very demanding work. It involves a bit of sleuthing, rigorous scientific assessment, and sensitive correspondence with researchers, institutions, and readers around the globe. It also requires resilience and tenacity – and time, as is discussed further in Q2, below. Our Publication Ethics editors abide by the Committee on Publication Ethics (COPE) guidance and are very careful to provide due respect to all contributors, due process to every case, and due consideration to every editorial decision. 

PLOS often receives questions about why we have taken so long to resolve issues raised to us or noted on Pubpeer a few years back. Many cases we are working through currently are from our ethics case backlog (discussed in Q3, below) which developed before we had a dedicated Publication Ethics team. The time between an initial notification and case resolution also reflects how the case has been prioritized by PLOS and the time required for PLOS and external contributors to complete various steps in the process.  

Below, I expand on some of these points and other frequently asked questions.. 

On behalf of our Publication Ethics team, I sincerely thank all who support PLOS in our efforts to address ethics and integrity issues, including authors, editors, reviewers, institutions, those who identify concerns and bring them to our attention, and our PLOS colleagues. 

If you have a concern about the integrity or reliability of PLOS content, please email the PLOS Publication Ethics team at pub-ethics@plos.org.

__________________________________________________________________________________________________________________

Frequently Asked Questions

Q1: How many cases does PLOS receive vs. resolve each year? 

In 2018-2021, our Publication Ethics team resolved >800 publication ethics cases and received >1100 new ones. Our team expanded in 2021 to address this disparity and create additional capacity for backlog case work. 

Q2: Why does it take so long to resolve publication ethics cases? 

The time between when we receive an issue and when we resolve the case reflects both where the case falls on our priority-ranked ‘to-do’ list, and how much time the case actually requires. 

Once we begin working on a post-publication case, it typically takes us ~4-8 months to bring it to resolution, although this varies considerably. This process involves a number of contributors each of whom takes anywhere from one week to multiple months to do their part (see table, below). 

When thinking about the time required for this work, it’s important to bear in mind that these aren’t just cases – they’re people. People whose work is being questioned, sometimes years after it was completed. People who may have retired or left academic research. People who are carving time out of their busy schedules to help PLOS on a volunteer basis. People who may be involved in investigating multiple concurrent integrity cases for an institution. People who are juggling a multitude of responsibilities – research, teaching, clinical practice, grant writing and review, not to mention family and other personal priorities. Over the last couple years, those involved in these cases have also been coping with the pandemic and the associated institutional closures, illnesses, and lockdown mandates. Although we work to keep cases moving along designated timelines, these personal considerations can impact and extend the process.

Case follow-up stage	Time required	Work required of PLOS staff or external contributors
Internal editorial assessment	2 weeks to multiple months	Evaluate multiple rounds of information and supporting files provided by contributors (author, editor, institution, etc.); discuss assessment and resolution within the Pub Ethics team and with the relevant journal’s Editorial lead(s)
Invite authors to comment on the concerns and obtain the underlying data, if needed	3 weeks to 3 months (or longer)	Depending on the nature of concerns, authors may need to identify, locate, and review archived lab records; reanalyze data; respond to scientific concerns; liaise with ethics committees
Consult subject matter experts (if needed)	1 week to multiple months	Identify editors or reviewers with the needed expertise and who are willing and able to provide input; complete a careful and rigorous review of the relevant materials; evaluate input received (multiple rounds of consultation are often needed)
Obtain institutional input (if needed)	1 month to multiple years	Establish communication with the relevant official(s); prioritize case at the institutional level; complete institutional proceedings (inquiry, review, and/or full investigation)
Editorial decision through publication of a notice	3-6 weeks	Evaluate all case records and come to a decision; prepare notice; notify authors of decision & give an opportunity to respond; production time (~2-4 weeks)
Appeals of editorial decisions	3 weeks – 6 months	Internal editorial assessment and discussion; depending on case details this step may also require consultation of subject matter experts, institutional involvement, and/or legal input

Q3: Why does PLOS have an ethics case backlog, and what are we doing to address it?

PLOS has always addressed concerns raised about our submissions and publications. For several years, these cases trickled in at a relatively low rate and were handled by the journals’ staff editors and Editorial Services teams, among their many other responsibilities. 

Then, in 2014-2016 our case numbers spiked: in addition to the steady stream of varied publication ethics concerns, PLOS received >400 post-publication image integrity cases. Those responding to these inquiries were quickly overwhelmed: the journal’s editors worked tirelessly, but could each handle only a couple cases at a time while juggling their other tasks. Hence, an ethics case backlog amassed quickly. 

At that time, we didn’t know if this was a one-off collection of cases, or if we would continue to receive cases at this unexpectedly high pace. We diverted internal staff resources to help where possible, but in essence PLOS was not yet equipped to respond to publication ethics issues at this scale. We needed more resources. In response, PLOS gradually built up a dedicated Publication Ethics team. This started with a single dedicated PLOS ONE editor in 2017, and has grown to a team of five Senior Editors serving all PLOS journals as a Center of Excellence. 

In late 2021 the PLOS Publication Ethics team began a multi-year project dedicated to addressing our ethics case backlog, some of which dates back to that 2014-16 surge. This is a large undertaking that we anticipate will require at least two years, but we are committed to ensuring that all issues which have been raised to us receive their due attention.

Q4: Why doesn’t PLOS take articles offline or post interim notices while they are under investigation?

We sincerely apologize for how long it is taking us to resolve our ethics case backlog and update the literature accordingly. We appreciate the importance of correcting the published record in a timely manner, and likewise we understand that it can negatively impact researchers and others to have problematic articles or unreliable results remain unmarked and unactioned in the published record. 

Contrary to requests we occasionally receive, we will not simply take an article offline while it is under investigation. Even for online-only publishers, the industry standard is to preserve the integrity and stability of the published record and to remove content only under rare circumstances.² (See also PLOS policy and COPE guidance.) PLOS is highly committed to transparency in research communication, and one aspect of this is ensuring an accurate and stable record of an article’s publication history.  

Interim notices can benefit readers while journals and/or institutions complete their investigations, but it is important to bear in mind that a substantial fraction of issues raised to PLOS do not ultimately warrant any formal editorial action. For example, concerns may not be verified per our editorial assessments, or may reflect differences of opinion rather than bona fide errors. 

PLOS’ current standard is to complete our follow-up work on most publication ethics cases before issuing editorial notices (if needed), and publish interim Expressions of Concern only in select circumstances for articles under investigation. However, we are currently considering an expanded framework of interim notice usage wherein we would routinely notify readers of editorially verified issues sooner. Whatever our course, we will continue to assess issues carefully before taking any public editorial action, and will ensure that our actions align with PLOS policies and COPE guidance. 

References

1. The Retraction Watch Database [Internet]. New York: The Center for Scientific Integrity. 2018. [Cited 10 February 2022]. Available from: http://retractiondatabase.org/.
2. COPE Council. COPE Retraction Guidelines – English. ©2019 Committee on Publication Ethics (CC BY-NC-ND 4.0). Version 2: November 2019. (https://doi.org/10.24318/cope.2019.1.4)