When you choose to publish with PLOS, your research makes an impact. Make your work accessible to all, without restrictions, and accelerate scientific discovery with options like preprints and published peer review that make your work more Open.

PLOS BLOGS The Official PLOS Blog

The potential butterfly effect of preregistered peer-reviewed research

Preregistration has the power to level up research communication. And it’s so easy.

Written by Lindsay Morton

Preregistration is the practice of formally registering a study design before experimentation and data collection begin. When deposited in a repository, a preregistered study design functions as a statement of intent laying claim to a certain investigatory pathway. The preregistered study design functions as an everyday tool for executing, reporting and repeating high-quality confirmatory investigations, as well as a stable methodological record capable of facilitating reproduction and reuse over the longer term. When practiced at a journal, two additional elements are added: peer review feedback, to help optimize the methods, and analytical approach; and provisional acceptance of the resulting research article when the investigation is completed, whatever the outcomes may be.

“Peer review of the registered study protocol provides an opportunity to improve the study design before starting to collect the data. That’s the most constructive time for the review to happen. And a published registered protocol is a guarantee for the authors that even null results will be published.” says Veronique Kiermer, Chief Scientific Officer at PLOS. Authors of preregistered research concur. As Jacqueline Scholl, Oxford University puts it, “”One major advantage compared to standard peer-review was that when reviewers made sensible suggestions about additional control variables to collect, we could actually implement this; rather than just thinking ‘we wished we had done that’.” 

In practical terms, neither form of preregistration outlined above is all that different from a standard research workflow. When conducting confirmatory research, investigators routinely define a hypothesis, develop methods, and choose how to collect and analyze data. These study design plans are frequently formalized as part of a grant application, and may well undergo peer review during the funding process as well.

Therefore, taking the minimal extra step of depositing a study design with a repository seems like such a small thing. A few buttons clicked, a document uploaded, and a check box or two. Initiating journal peer review earlier in the process is a similarly a minor shift; a simple reorganization of existing processes into a more efficient configuration. And yet, on the individual level, and especially in aggregate, these modest tweaks to the peer review and publication workflow have the potential to exert an outsized impact on research and scientific communication.

Privately depositing a study design with a repository helps researchers stay accountable to themselves, reducing the risk of behaviors like p-hacking, selective publication, or suppressing negative and null research outcomes. Researchers who choose to make their study designs public from the time of deposition help to avoid unnecessary repeated studies, unintentional complementary research, and scooping, freeing up their fellow researchers to pursue different lines of inquiry and improving efficiency in the field. When private repository study designs become public upon publication of a research article, their existence serves to bolster the final work, demonstrating integrity on the part of the authors, and supporting trust among readers. Finally, retaining a permanent methodological record documenting the data collection and analysis as it was actually planned and performed ensures future reproducibility as almost nothing else can.

Refocusing journal peer review on the study design phase exerts more and greater downstream changes. Peer review that focuses on evaluating significance of the research question, the methods and analytical approach before work begins, has the power to shape stronger, more rigorous and more creative research. Making an editorial decision while results are still unknown minimizes the potential impacts confirmation bias and impact bias, taking science communication back to its roots, with an emphasis on quality, rigor, and a pure intellectual curiosity. As Kiermer explains, “Preregistration and peer review of the study protocol with a journal is a way to tackle publication bias. As long as the protocol is followed, or any deviations explained, it’s a guarantee for the author that the results will be published, even if they don’t confirm their hypothesis.”

In combination, all of these factors contribute to a more complete and efficient scientific record, replete with studies exploring important hypotheses, performed to the very highest technical standards, and free from the distorting influence of impact-chasing, ego, and bias. A scientific record that is both demonstrably trustworthy, and widely trusted. And with that, there is no telling where science might go, or how quickly.

Discussion
  1. Re: “Privately depositing a study design with a repository helps researchers stay accountable to themselves, reducing the risk of behaviors like p-hacking, selective publication, or suppressing negative and null research outcomes.”

    This will make it almost impossible for biologically uninformed theorists to link their experimental evidence to stupid theories.

    For comparison, I’ve watched as the research of all intelligent serious scientists on light-activated microRNAs was linked from the ATP-dependent Creation of RNA to former President Trump’s claim on 4/23/20 that sunlight and humidity weaken COVID19 via more than 138,000 indexed published works that mention “microRNAs.”

    Now, others can learn about the facts that link the energy-dependent microRNA-mediated construction of the cell wall to protection from all virus-driven pathology via the STEM toy “Cellulose” for ages 14 plus. See: https://www.kickstarter.com/projects/geniusgames/cellulose-a-plant-cell-biology-game

  2. This is a very interesting proposal and approach. Two things in particular strike me. First of all, isn’t this going to overburden the reviewer pool even more? But perhaps even more important: Is a journal seriously going to guarantee publication because one has been through peer review with the proposal? Is there an “acceptance” stage, i.e. could the peer review of the proposal in principle lead to rejection? But, again, I like the vision. I’m just wondering about these points. Looking forward to your reply!
    Curt Rice, Rector, Norwegian University of Life Sciences

    1. Hi Curt, thank you for these thoughtful questions!

      Taking them in order…

      Will preregistration overburden the reviewer pool even more?

      From the reviewers’ point of view, the effort involved in assessing a preregistered research article should be similar to that of reviewing a standard research article–even though the review does take place in two discrete stages. In this model the bulk of constructive feedback takes place during the study design phase, while the review of the final research article focuses on confirming the authors followed their approved protocol, and assessing the results and conclusions for thoroughness, accuracy, and clarity.

      However, there is the possibility that, because of the longer period of time between study design and finished article as compared with the revision rounds of a standard research article submission, the same reviewers may not be available for both stages. That could make the assessment more challenging for reviewers coming to the full research article without first having seen the proposed study design. Something to watch for in the publishing system metadata.

      Do journals seriously guarantee publication based on peer review and provisional acceptance of the proposal?

      Yes! Provided that the authors adhere to the peer reviewed and approved study design, and uphold all applicable standards for ethics and reporting, they can rest assured that the article will go on to be formally accepted and published once the results are in. You can read a bit more on how the process has been implemented at PLOS ONE here.

      Importantly, preregistered studies can include post hoc analysis in addition to the analysis outlined in the original study design, if warranted.

  3. Know how the potential butterfly effect of preregistered peer-reviewed research by reading this blog. thanks for such valuable information.

Leave a Reply

Your email address will not be published. Required fields are marked *


Add your ORCID here. (e.g. 0000-0002-7299-680X)

Related Posts
Back to top