Abstract: Concerns over reproducibility in science extend to research using existing healthcare data; many observational studies investigating the same topic produce conflicting results, even when using the same data. To address this problem, we propose a paradigm shift. The current paradigm centers on generating one estimate at a time using a unique study design with unknown reliability and publishing (or not) one estimate at a time. The new paradigm advocates for high-throughput observational studies using consistent and standardized methods, allowing evaluation, calibration, and unbiased dissemination to generate a more reliable and complete evidence base. We demonstrate this new paradigm by comparing all depression treatments for a set of outcomes, producing 17,718 hazard ratios, each using methodology on par with state-of-the-art studies. We furthermore include control hypotheses to evaluate and calibrate our evidence generation process. Results show good transitivity and consistency between databases, and agree with four out of the five findings from clinical trials. The distribution of effect size estimates reported in literature reveals an absence of small or null effects, with a sharp cutoff at p = 0.05. No such phenomena were observed in our results, suggesting more complete and more reliable evidence.



Bio: Marc A. Suchard is a Professor in the Departments of Biostatistics, of Biomathematics and of Human Genetics in the UCLA Fielding School of Public Health and David Geffen School of Medicine at UCLA. He earned his Ph.D. in biomathematics from UCLA in 2002 and continued for a M.D. degree which he received in 2004. Dr. Suchard is a leading Bayesian statistician who focuses on inference of stochastic processes in molecular epidemiology of infectious diseases. His training in both medicine and applied probability help to bridge the gap of understanding between statistical theory and clinical practicality. Dr. Suchard has been awarded several prestigious statistical awards such as the 2003 Savage Award, the 2006 and 2011 Mitchell Prizes, as well as a 2007 Alfred P. Sloan Research Fellowship in computational and molecular evolutionary biology, and a 2008 Guggenheim Fellowship to further computational statistics. Finally, he received the 2011 Raymond J. Carroll Young Investigator Award and the 2013 Committee of Presidents of Statistical Societies (COPSS) Presidents’ Award for outstanding contributions to the statistics profession by a person aged 40 or under.