When you choose to publish with PLOS, your research makes an impact. Make your work accessible to all, without restrictions, and accelerate scientific discovery with options like preprints and published peer review that make your work more Open.

PLOS BLOGS The Official PLOS Blog

PLOS Publication Ethics: A frank discussion on handling difficult cases

Author: Renee Hoch, Senior Editor & Manager of the PLOS Publication Ethics Team

Since 2003, PLOS has published nearly 300,000 research articles, contributing a tremendous body of knowledge to the scientific corpus. However, our roles in scientific communications do not end at the time of publication. PLOS, like many other scholarly publishers, has a Publication Ethics team dedicated to addressing ethics and integrity concerns raised about PLOS content, many of which arise after publication. 

PLOS Publication Ethics receives a broad array of inquiries, ranging from correctable reporting errors to more serious issues, including data analysis, data integrity, or reporting concerns, study design flaws, authorship and data ownership disputes, and highly sensitive matters involving research ethics, publication ethics, and potential misconduct. Altogether, PLOS has been notified of post-publication ethics or integrity concerns for <1% of our published research articles, and thus far we have issued 380 retractions (~0.13% of PLOS research articles). We are not alone: the Retraction Watch Database1 includes over 17,000 retracted research articles.  

Why are these issues not identified before publication

All PLOS research articles are peer-reviewed, but peer review has its limitations. A manuscript is typically evaluated by 2-4 people during pre-publication peer review, whereas after publication it is available to a much broader audience, including collaborators, other researchers, those exploring ethics and integrity in the research literature, and even the press and general public. As open access publications, PLOS articles are freely available for all to read, and each reader views the work through their unique perspective. With many more eyes on the content post-publication, it is perhaps unsurprising that issues may come to light after having slipped unnoticed through peer review.  

Furthermore, some issue types (e.g. data ownership concerns) cannot be identified from the article content alone or involve people who would not typically be involved in peer review. We do not expect these types of concern to be detectable pre-publication.

Editorial actions are not indicators of misconduct

While social media and news outlets often highlight egregious cases and/or those involving misconduct, much of what the PLOS Publication Ethics team sees may have arisen due to honest errors. Importantly, it is beyond the journal’s or publisher’s remit to adjudicate on intent or personal culpability. Regardless of who was responsible, regardless of whether issues arose due to honest errors or deliberate misconduct, and regardless of when issues are noticed, PLOS has a responsibility to take action as needed to ensure the integrity and reliability of our publications. We take this responsibility very seriously. 

What it takes to resolve publication ethics cases

Each publication ethics case – even for seemingly straightforward issues – takes its own path and involves multiple contributors. This is very demanding work. It involves a bit of sleuthing, rigorous scientific assessment, and sensitive correspondence with researchers, institutions, and readers around the globe. It also requires resilience and tenacity – and time, as is discussed further in Q2, below. Our Publication Ethics editors abide by the Committee on Publication Ethics (COPE) guidance and are very careful to provide due respect to all contributors, due process to every case, and due consideration to every editorial decision. 

PLOS often receives questions about why we have taken so long to resolve issues raised to us or noted on Pubpeer a few years back. Many cases we are working through currently are from our ethics case backlog (discussed in Q3, below) which developed before we had a dedicated Publication Ethics team. The time between an initial notification and case resolution also reflects how the case has been prioritized by PLOS and the time required for PLOS and external contributors to complete various steps in the process.  

Below, I expand on some of these points and other frequently asked questions.. 

On behalf of our Publication Ethics team, I sincerely thank all who support PLOS in our efforts to address ethics and integrity issues, including authors, editors, reviewers, institutions, those who identify concerns and bring them to our attention, and our PLOS colleagues. 

If you have a concern about the integrity or reliability of PLOS content, please email the PLOS Publication Ethics team at pub-ethics@plos.org.

__________________________________________________________________________________________________________________

Frequently Asked Questions

Q1: How many cases does PLOS receive vs. resolve each year? 

In 2018-2021, our Publication Ethics team resolved >800 publication ethics cases and received >1100 new ones. Our team expanded in 2021 to address this disparity and create additional capacity for backlog case work. 

Q2: Why does it take so long to resolve publication ethics cases? 

The time between when we receive an issue and when we resolve the case reflects both where the case falls on our priority-ranked ‘to-do’ list, and how much time the case actually requires. 

Once we begin working on a post-publication case, it typically takes us ~4-8 months to bring it to resolution, although this varies considerably. This process involves a number of contributors each of whom takes anywhere from one week to multiple months to do their part (see table, below). 

When thinking about the time required for this work, it’s important to bear in mind that these aren’t just cases – they’re people. People whose work is being questioned, sometimes years after it was completed. People who may have retired or left academic research. People who are carving time out of their busy schedules to help PLOS on a volunteer basis. People who may be involved in investigating multiple concurrent integrity cases for an institution. People who are juggling a multitude of responsibilities – research, teaching, clinical practice, grant writing and review, not to mention family and other personal priorities. Over the last couple years, those involved in these cases have also been coping with the pandemic and the associated institutional closures, illnesses, and lockdown mandates. Although we work to keep cases moving along designated timelines, these personal considerations can impact and extend the process.

Case follow-up stageTime requiredWork required of PLOS staff or external contributors
Internal editorial assessment2 weeks to multiple monthsEvaluate multiple rounds of information and supporting files provided by contributors (author, editor, institution, etc.); discuss assessment and resolution within the Pub Ethics team and with the relevant journal’s Editorial lead(s)
Invite authors to comment on the concerns and obtain the underlying data, if needed 3 weeks to 3 months (or longer)Depending on the nature of concerns, authors may need to identify, locate, and review archived lab records; reanalyze data; respond to scientific concerns; liaise with ethics committees
Consult subject matter experts (if needed)1 week to multiple monthsIdentify editors or reviewers with the needed expertise and who are willing and able to provide input; complete a careful and rigorous review of the relevant materials; evaluate input received (multiple rounds of consultation are often needed)
Obtain institutional input (if needed)1 month to multiple yearsEstablish communication with the relevant official(s); prioritize case at the institutional level; complete institutional proceedings (inquiry, review, and/or full investigation) 
Editorial decision through publication of a notice 3-6 weeksEvaluate all case records and come to a decision; prepare notice; notify authors of decision & give an opportunity to respond; production time (~2-4 weeks)
Appeals of editorial decisions3 weeks – 6 monthsInternal editorial assessment and discussion; depending on case details this step may also require consultation of subject matter experts, institutional involvement, and/or legal input 

Q3: Why does PLOS have an ethics case backlog, and what are we doing to address it?

PLOS has always addressed concerns raised about our submissions and publications. For several years, these cases trickled in at a relatively low rate and were handled by the journals’ staff editors and Editorial Services teams, among their many other responsibilities. 

Then, in 2014-2016 our case numbers spiked: in addition to the steady stream of varied publication ethics concerns, PLOS received >400 post-publication image integrity cases. Those responding to these inquiries were quickly overwhelmed: the journal’s editors worked tirelessly, but could each handle only a couple cases at a time while juggling their other tasks. Hence, an ethics case backlog amassed quickly. 

At that time, we didn’t know if this was a one-off collection of cases, or if we would continue to receive cases at this unexpectedly high pace. We diverted internal staff resources to help where possible, but in essence PLOS was not yet equipped to respond to publication ethics issues at this scale. We needed more resources. In response, PLOS gradually built up a dedicated Publication Ethics team. This started with a single dedicated PLOS ONE editor in 2017, and has grown to a team of five Senior Editors serving all PLOS journals as a Center of Excellence. 

In late 2021 the PLOS Publication Ethics team began a multi-year project dedicated to addressing our ethics case backlog, some of which dates back to that 2014-16 surge. This is a large undertaking that we anticipate will require at least two years, but we are committed to ensuring that all issues which have been raised to us receive their due attention.

Q4: Why doesn’t PLOS take articles offline or post interim notices while they are under investigation?

We sincerely apologize for how long it is taking us to resolve our ethics case backlog and update the literature accordingly. We appreciate the importance of correcting the published record in a timely manner, and likewise we understand that it can negatively impact researchers and others to have problematic articles or unreliable results remain unmarked and unactioned in the published record. 

Contrary to requests we occasionally receive, we will not simply take an article offline while it is under investigation. Even for online-only publishers, the industry standard is to preserve the integrity and stability of the published record and to remove content only under rare circumstances.2 (See also PLOS policy and COPE guidance.) PLOS is highly committed to transparency in research communication, and one aspect of this is ensuring an accurate and stable record of an article’s publication history.  

Interim notices can benefit readers while journals and/or institutions complete their investigations, but it is important to bear in mind that a substantial fraction of issues raised to PLOS do not ultimately warrant any formal editorial action. For example, concerns may not be verified per our editorial assessments, or may reflect differences of opinion rather than bona fide errors. 

PLOS’ current standard is to complete our follow-up work on most publication ethics cases before issuing editorial notices (if needed), and publish interim Expressions of Concern only in select circumstances for articles under investigation. However, we are currently considering an expanded framework of interim notice usage wherein we would routinely notify readers of editorially verified issues sooner. Whatever our course, we will continue to assess issues carefully before taking any public editorial action, and will ensure that our actions align with PLOS policies and COPE guidance. 

References

  1. The Retraction Watch Database [Internet]. New York: The Center for Scientific Integrity. 2018. [Cited 10 February 2022]. Available from: http://retractiondatabase.org/.
  2. COPE Council. COPE Retraction Guidelines – English. ©2019 Committee on Publication Ethics (CC BY-NC-ND 4.0). Version 2: November 2019. (https://doi.org/10.24318/cope.2019.1.4
Discussion
  1. How much of the delay in the huge number of your questioned image cases could be solved if you simply had the corresponding author sign a prepublication agreement saying all authors would accept immediate retraction if it was found, after any allegation (credible or not) that the primary data was missing?

    This painless initiative is obviously your choice, but one well within your purview. It would protect the publisher’s investment in the paper, inspire all authors to insist on first seeing/reviewing the primary data (universally, in ORI cases they hadn’t) and by default motivate their host institutions for developing data retention policies (which involve only pennies). It would be a savvy market ploy to set PLOS standards ahead of its competitors and, best of all, shorten correction of the literature.

    1. PLOS has always been a strong advocate for open science and open data sharing, and indeed post-publication questions can be more swiftly and positively resolved if the data are readily available. Posting primary data publicly with the article or in a stable data repository can help to bypass case delays due to the time required to find, unarchive, and organize data from historical lab records. It can also ensure that data will remain available to address any future queries even if researchers leave the laboratory, there are technical issues with data storage (aka computer crashes), or questions are raised outside the time period during which institutions or funders require data to be retained.

      In principle, the crux of your suggestion (minus the retraction outcome/penalty) is already in place at PLOS. Since we updated our Data Availability policy in 2014, PLOS has required authors to make underlying data publicly available without restriction at the time of publication, unless the authors do not own the data or cannot share the data publicly for legal or ethical reasons. In 2019, PLOS ONE and PLOS Biology added policies that specifically require provision of raw blot and gel images.

      While it is within PLOS’ purview to develop and adjust our internal policies, we abide by Committee on Publication Ethics (COPE) guidance, including the COPE Retraction Guidelines. According to these guidelines, data unavailability in itself does not warrant retraction. With that said, data unavailability can hinder our ability to clarify concerns about published results, and for this reason the availability of primary data can and does impact our editorial decisions in cases involving data or image integrity concerns.

      Finally, please note that any future updates to our Data Availability policy would not impact our backlog cases. PLOS publications are required to comply with the Data Availability policy that is/was in place when the article was submitted, and so a change in policy would not impact case outcomes for articles submitted before the new policy’s implementation date. We require authors to confirm at submission their ability to comply with the current Data Availability policy, and as is discussed here PLOS publishes Expressions of Concern when post-publication questions about noncompliance with the policy cannot be positively resolved.

Leave a Reply

Your email address will not be published. Required fields are marked *


Add your ORCID here. (e.g. 0000-0002-7299-680X)

Related Posts
Back to top