Socio-Epistemic Bubbles and Tacit Confidence in Randomized Controlled Trials

What the paper says

The paradigm of scientific medicine is among the most influential epistemic shifts in the past century, wherein randomized controlled trials (RCTs) represent the impartial arbiter of medical claims. Nevertheless, not all RCTs agree, and systematic reviews are invoked to reconcile them. We theorize how tacitness-beliefs, implicit assumptions, expectations, heuristics, unrecorded routines-accumulates within 'socio-epistemic bubbles', continuous regions of social density that decrease diversity and increase unwarranted certainty about healthcare studied by RCTs. To assess our theory, we analyze 20,117 meta-analyses extracted from 1,962 Cochrane systematic reviews. We find that being closer within 'social space' inscribed by scientific collaboration increases agreement across RCTs. Our analysis suggests that this amplified certainty can drive premature convergence affecting medical practice and population health. By increasing researchers' ability to perform an experiment in the way required to achieve an expected result, socio-epistemic bubbles represent the dark matter of the experimental process often obscured. Our findings imply hidden limitations associated with unmeasured social influence across the policy sciences, through which conflicting claims perpetuate and highlight the necessity of accounting for them to improve collective certainty.