Escaping science’s paradox

Interesting article by Stuart Buck from Arnold Ventures, which has been a major funder in open and replicable science.

Science has two stark problems: replication and innovation. Many scientific findings aren’t reproducible. That is to say, you can’t be sure that another study or experiment on the same question would get similar results. At the same time, the pace of scientific innovation could be slowing down.

Does attempting to solve one problem make the other worse? Many have argued that policies seeking to avoid reproducibility issues will create a constrictive atmosphere that inhibits innovation and discovery.

Indeed, top policymakers are worried about just this. Along with other prominent philanthropists and academics, I attended a White House meeting on scientific reproducibility early in 2020 (just before COVID-19 really hit). One of the key questions on a sheet of paper that the White House Office of Science and Technology Policy circulated for discussion was whether a tradeoff existed: Would efforts to improve reproducibility risk harming the creativity and innovation of federally-funded research?

I do not think there’s a contradiction between reproducibility and innovation. Contrary to common belief, we can improve both at once – by incentivizing failed results, and by funding “Red Teams” that would aim to refute existing dogma or would be entirely outside it.

Improving reproducibility and innovation isn’t easy, to be sure. But science policy and science funders could do both at once by demanding more null results, and by substantially funding efforts to contradict groupthink and confirmation bias. And this would help all of society get more value out of the many billions of dollars that we collectively spend on science every year.

1 Like

This article talks more about that:

Also, from this write up:

  • The Laura and John Arnold Foundation is the biggest funder of meta-research we’re currently aware of which specifically has a focus area on research integrity[73]. From 2011-2017 the Arnold foundation has given about $81 million to meta-research causes like the Meta-Research Innovation Center at Stanford (METRICS) [74], the Centre for Open Science[75], and several other initiatives[76].

Note, the Research Integrity area on the Arnold Ventures site is no longer active, but it looks like this is now just part of their Research area.

@Gavin: hey, the concept of red teams make another appearance… :slight_smile:

@Dasapta_Erwin_Irawan and @dasaptaerwin: remember when i send that @briannosek 's article to show how fortunate osf is, along with other related ones to highlight how difficult it is to get funding for similar initiatives in indonesia.

hence my last year’s presentation abstract for open access week, and another email which for some reason did get any reply by nick shockley and friends at sparc and opencon, till now. perhaps because of my close association with jon?

ah, c’est la vie. :slight_smile: (on to the demonization of muslims in france! :frowning: )