A research article is an orderly summation of a complex and circuitous process. It is characterized by detailed planning, iterative trial and…
Written by Lindsay Morton
In scientific communications, methods are finally getting their due. Tools for better-communicating methods are everywhere these days—from new reporting standards and methods-specific article types, to dedicated methods journals and purpose-built repository platforms. But so far, no single solution has enjoyed wide adoption or been generally acknowledged as best practice.
Now, new data gathered by PLOS, with the support of protocols.io and TCC Africa, sheds light on how researchers view methods, and lends insight into their motivations and behaviors when it comes to methods-sharing. Over 1,000 researchers completed the survey. Respondents were concentrated primarily in the Life and Health Sciences, and tended to be more senior in their careers. Read on for highlights, or skip straight to the preprint for in-depth details.
Takeaway #1 – Established methods-sharing norms are insufficient
Researchers report that, in general, for articles in their field, the information contained in research article methods sections is only moderately adequate for evaluating findings, and inadequate for reproduction and reuse. Specifically:
- Only 74% of respondents reported that the methods section alone is usually adequate to evaluate the findings of the article.
- Only 42% reported that the methods section is usually adequate to reproduce results.
- And 47% that it is usually adequate to reuse or extend the method in a different context.
Takeaway #2 – Researchers see methods sharing as important
Researchers care about methods-sharing. Nearly all respondents (95%) rate sharing novel methods as important or highly important. A majority (60%) rate methods-sharing as important even if the methods themselves are straightforward and widely used.
Takeaway #3 – When it comes to their specific goals, researchers aren’t satisfied
Researchers were asked to rank specific reasons or goals for methods-sharing in two ways:
- Importance of the goal
- Satisfaction with their own ability to achieve the goal
In general, goals relating to usability of methods are of paramount importance to researchers, and satisfaction with their ability to achieve this is moderate. Goals relating to access and availability are seen as slightly less important, but researchers also report a wider discrepancy between importance and satisfaction.
Takeaway #4 – The main blockers to methods-sharing are practical
A minority of respondents (33%) report having prior experience sharing methods publicly, while 56% have shared methods privately upon request. Given the high value these researchers place on methods, and their assessment of the adequacy of research article methods sections in general, we have to wonder—what’s stopping them from sharing more?
The major blockers to methods sharing are practical considerations. Over a third of respondents rate the time it takes to prepare methods for public consumption (40%) and not knowing where to share (34%) as significant or highly significant blockers.
Of slightly less significance are concerns that sharing may not be worth their time, either because few people would use the information (30%) or because others might lay claim to their ideas (30%). Few researchers (14%) express concern about exposing their research to scrutiny or criticism.
What do we do with this information?
Reflecting on the survey, Lenny Teytelman, CEO and co-founder of protocols.io writes, “This is the first method-focused survey that I am aware of. When we had the idea for protocols.io a decade ago, we were acting on our intuition. It is great to see the results of the survey, validating some of our core assumptions, but more importantly, illuminating some ideas that we should be considering and trying.”
It’s our hope that this survey and others like it can help to identify unmet needs in scholarly communications, and ultimately lead to more effective tools and best practices. We also hope that interested meta-researchers will dig into the data, draw their own conclusions, and help to direct future research in this area.