Strengthen science by funding living evidence synthesis

A recent high-profile paper suggesting that science is becoming dramatically less disruptive set the academic world abuzz. One interpretation of this finding — though by no means the only one — is that the exponential growth of the scientific literature has led researchers to build off older and increasingly narrow slices of the research landscape. Indeed, an earlier study showed that as fields grow, attention consolidates around foundational papers at the expense of newer work. In the busiest fields, the top 1% of cited papers remain frozen in time, while virtually no new research enters their ranks.

Engagement with a broad range of knowledge is critical for innovation. So although “scientists are increasingly citing famous work” sounds somewhat innocuous, the underlying knowledge overload poses a genuine threat. If our processes fail to adapt, a continued flood of papers could lead to further slowing of transformative science, more redundant studies, less efficient targeting of knowledge gaps, and decreased uptake of cutting-edge research.

A powerful approach to fixing this problem exists, but it is currently underutilized. Living evidence is a framework for building and maintaining an up-to-date and rigorous synthesis of knowledge within a scientific area. Depending on the specific need, this model can be applied to enhance understanding for research and innovation or facilitate translation of research into policy and practice.

While the idea of living evidence synthesis has been around for a while, it came into its own during the Covid-19 pandemic when thousands of papers were being published every day and important evidence-based decisions needed to be made quickly. Products like this living systematic review on drug treatments for Covid-19 were enormously valuable for aggregating complex, fast-moving, and mixed-quality science.

The living evidence model can provide immense value to countless fields, but is yet to be widely implemented beyond health care. That’s a shame. Consistent funding for up-to-date, expert-led syntheses could help tame the burden of knowledge, increase scientific efficiency, improve translation and policy implementation, and provide often-elusive funding to early-career scientists.

Government and philanthropic funders of science should take this opportunity seriously, and none is better positioned to do so than the National Institutes of Health — the largest funder of biomedical science in the world. By defining living evidence products as “research resources” and funding them under the existing grant mechanism known as the R24, the NIH could unlock the value of up-to-date knowledge synthesis without needing to create a new mechanism.

Scientific and societal benefits

We see five important benefits that greater production of living evidence could have for science and society.

Promote transformative science. Scientific progress relies on researchers’ ability to make connections between disparate and cross-disciplinary ideas. With an increasingly overwhelming and complex body of literature, better processes are needed to allow scientists to engage with a broad range of knowledge. The concise, high-fidelity representation of current knowledge provided by living evidence can help. By aiding researchers in their efforts to build on the latest science across diverse domains, this approach would facilitate the novel connections that often underpin innovation.

Tame the burden of knowledge for new researchers. As information accumulates, the amount of time needed to reach the cutting edge and make novel contributions — often called the burden of knowledge — may stall young scientists’ innovative potential and make research less efficient. Early-career scientists often lament the difficulty of gaining deep, intuitive knowledge of their field as it grows and changes. By providing scientists of all stripes a unified resource for getting — and staying — on top of their fields, living evidence could speed up the education-to-innovation pipeline.

Direct scientists toward critical knowledge gaps. Effective living evidence products can provide much more than a summary of the existing literature. They can leverage the broad and deep insights gained by rigorous and up-to-date synthesis to describe gaps in existing knowledge, raise neglected or unanswered questions, propose novel combinations of topics, and identify claims in need of replication. This could range from an author’s summary of implications and conclusions to more formal scientific road-mapping or research prioritization exercises.

Support evidence-based decisions and policymaking. In areas where practitioners, policymakers, and consumers rely on research to make evidence-based decisions, the risks of outdated knowledge go beyond scientific inefficiency. Without access to an up-to-date synthesis of risks and benefits, for example, one can imagine that patients seeking relief from severe mental illness may choose to self-treat with unregulated psychedelics based on overhyped studies. Living evidence can also enable effective knowledge translation by creating an opportunity for active partnerships between researchers, businesses, governments, and the public.

Create a new opportunity for early-career funding. For decades, the proportion of NIH awards given to early-career researchers has declined while the average age of recipients climbs. New policies to boost new and early-stage investigators have barely moved the needle. Grants for knowledge synthesis, though not explicitly targeted at young scientists, would be ideal for these scholars, who tend to be steeped in the literature but don’t yet have funding for research projects. And for roughly one-third the budget of an R01 (the NIH’s flagship research grant), a living evidence grant could provide new faculty enough support to grow their domain knowledge, build name recognition, identify research programs, hire students, and submit project proposals.

Obstacles to adoption

A living evidence initiative is a scientific project requiring committed person time and resources. Funders like the Wellcome Trust are investing in living evidence for specific domain areas, but there is not yet large-scale grant funding that would enable researchers and partners to build out the model across science. Creating these funding opportunities is a critical next step to optimizing and scaling this model for the research community.

Even with grants to provide funding and prestige for living evidence work, some might worry that this type of research wouldn’t attract qualified scientists. However, the growth of the evidence synthesis field has shown that early-to-mid career scientists — who are familiar with the new work in their field, and who tend to be technically skilled — will be interested in these opportunities. More senior scientists with well-funded research groups may also find synthesis grants appealing: The funding would provide flexibility, while the process of mapping the latest literature could provide educational and productivity benefits for team members.

The methodological challenges faced by static synthesis will remain important. Yet we believe the dynamic nature of a living evidence approach would make them more tractable. For example, regular updates mean that living reviews are well-placed to remove erroneous or retracted papers. There is also an opportunity to engage readers with comment sections, where flaws and recommendations could be discussed openly and incorporated directly into the primary text. And though it is imperfect, grant review could improve quality by allowing experts to evaluate the planned synthesis approach.

Taking the next steps

Aggregation, synthesis, and road-mapping are key tools for biomedical research and practice. To optimize these processes, more attention and funding are needed for scientist-led living evidence work. The NIH is uniquely positioned to provide both.

The NIH’s 27 institutes and centers could support living evidence without any institutional reform or new mechanisms by leveraging the existing “research resources” R24 grant. Across the NIH, this grant has been used to fund a wide variety of products, including data, infrastructural, social, and technological resources. Knowledge resources are a natural addition to this list.

Individual institutes could implement funding for living evidence according to their own needs. They could add a pilot project covering a few key topics; include new text about knowledge resources in existing requests for proposals; or tether companion synthesis awards to large-scale programs like the Helping to End Addiction Long-term Initiative.

Regardless of the specific form, supporting the production and maintenance of living evidence could benefit science, scientists, and society.

Jordan Dworkin is the program lead for metascience at the Federation of American Scientists, a nonprofit, nonpartisan policy research organization working to develop and implement innovative ideas in science and technology. Julian Elliott is the co-founder and CEO of the Future Evidence Foundation, a nonprofit organization that is working to change the way the world creates and uses trustworthy knowledge.