Reproducibility and robustness of economics and political science research.

Brodeur A., Mikola D., Cook N., Fiala L., Brailey T., Briggs R., de Gendre A., Dupraz Y., Gabani J., Gauriot R., Haddad J., Lima G., Ankel-Peters J., Dreber A., Campbell D., Kattan L., Marino Fages D., Mierisch F., Sun P., Wright T., Connolly M., Hoces de la Guardia F., Johannesson M., Miguel E., Vilhuber L., Abarca A., Acharya M., Adjisse SS., Akhtar A., Ramirez Lizardi EA., Albrecht S., Andersen SN., Andlib Z., Arrora F., Ash T., Bacher E., Bachler S., Bacon F., Bagues M., Balogh T., Batmanov A., Barschkett M., Basdil BK., Baxa J., Becker SO., Beeder M., Beland L-P., Bello A-H., Markovits DB., Benjamin G., Bergeron T., Blimpo MP., Binetti M., Bonander C., Bonneau J., Borbáth E., Borgen N., Borgen ST., Borowsky J., Brini E., Brown M., Brun M., Bruns S., Buliskeria N., Calef A., Cameron A., Campa P., Campos-Rodríguez S., Cantone GG., Carpena F., Carter PJ., Dower PC., Castek O., Caviglia-Harris J., Strand GC., Chen S., Chzhen SI., Chung J., Collins J., Coppock A., Cordeau H., Couillard B., Crechet J., Crippa L., Cui J., Czymara C., Daarstad H., Dao DC., Dao D., Schmandt MD., de Linde A., De Melo L., Deer L., De Vera M., Dimitrova V., Dollbaum JF., Dollbaum JM., Donnelly M., Huynh LDT., Dumbalska T., Duncan J., Duong KT., Duprey T., Dworschak C., Ellingsrud S., Elminejad A., Eissa Y., Erhart A., Etingin-Frati G., Fatemipour E., Federice A., Feld J., Fenig G., Firouzjaeiangalougah M., Fleisje E., FortiFriter-Chouinard A., Engel JF., Fréchet N., Fortier R., Fries T., Frith MJ., Galipeau T., Gallegos S., Gangji A., Gao X., Garnache C., Gáspár A., Gavrilova E., Ghosh A., Gibney G., Gibson G., Godager G., Goff L., Gong D., González J., Gretton JD., Griffa C., Grigoryeva I., Grøtting M., Guntermann E., Guo J., Gugushvili A., Habibnia H., Häffner S., Hall JD., Hammar O., Kordt AH., Hashimoto B., Hartley JS., Hausladen CI., Havránek T., He H., Hepplewhite M., Herrera-Rodriguez M., Heuer F., Heyes A., Ho ATY., Holmes J., Holzknecht A., Hsu Y-HD., Hu S-H., Huang Y-S., Huebener M., Huber C., Huynh KP., Irsova Z., Isler O., Jakobsson N., Jananji R., Jayalath TA., Jetter M., John J., Forshaw RJ., Juan F., Kadriu V., Karim S., Kelly E., Hoang Dang DK., Khushboo T., Kim J., Kjellsson G., Kjelsrud A., Kotsadam A., Korpershoek J., Krashinsky L., Kundu S., Kustov A., Lalayev N., Langlois A., Laufer J., Lee-Whiting B., Leibing A., Lenz G., Levin J., Li P., Li T., Lin Y., Listo A., Liu D., Lu X., Lukmanova E., Luscombe A., Lusher LR., Lyu K., Ma H., Mäder N., Makate C., Malmberg A., Maitra A., Mandas M., Marcus J., Margaryan S., Márk L., Martignano A., Marsh A., Masetto I., McCanny A., McManus E., McWay R., Metson L., Kinge JM., Mishra S., Mohnen M., Moeller J., Montambeault R., Montpetit S., Morin L-P., Morris T., Moser S., Motoki FYS., Muehlenbachs L., Musulan A., Musumeci M., Nabin M., Nchare K., Neubauer F., Nguyen QMP., Nguyen T., Nguyen-Tien V., Niazi A., Nikolaishvili G., Nordstrom A., Nüß P., Odermatt A., Olson M., Øien H., Ölkers T., Oliver I Vert M., Oral E., Oswald C., Ousman A., Özak Ö., Pandey S., Pavlov A., Pelli M., Penheiro R., Park R., Martel EP., Petrovičová T., Phan L., Prettyman A., Procházka J., Putri A., Quandt J., Qiu K., Thi Nguyen LQ., Rahman A., Rea CH., Reiremo A., Renée L., Richardson J., Rivers N., Rodrigues B., Roelofs W., Roemer T., Rogeberg O., Rose J., Roskos-Ewoldsen A., Rosmer P., Sabada B., Saberian S., Salamanca N., Sator G., Scates D., Schlüter E., Sells C., Sen S., Sethi R., Shcherbiak A., Sogaolu M., Soosalu M., Sørensen EØ., Sovani M., Spencer N., Staubli S., Stans R., Stewart A., Stips F., Stockley K., Strobel S., Struby E., Tang JP., Tanrisever I., Yang TT., Tastan I., Tatić D., Tatlow B., Seuyong FT., Thériault R., Thivierge V., Tian W., Toma F-M., Totarelli M., Tran V-A., Truong H., Tsoy N., Tuzcuoglu K., Ubfal D., Villalobos L., Walterskirchen J., Wang JT-Y., Wattal V., Webb MD., Weber BS., Weisser R., Weng W-C., Westheide C., White K., Winter J., Wochner T., Woerman M., Wong J., Woodard R., Wroński M., Yazbeck M., Yang GC., Yap L., Yassin K., Ye H., Yoon JY., Yurris C., Zahra T., Zaneva M., Zayat A., Zhang J., Zhao Z., Zhong Y.

Science aspires to be cumulative. Reproducibility efforts strengthen science by testing the reliability of published findings, promoting self-correction, and informing policy-making1. Computational reproductions, whereby independent researchers reproduce the results of published studies, are an essential diagnostic tool2-10. Such efforts should have greater visibility11-16. However, little social science reproduction and robustness has been conducted at scale10,13,17-23. Here we reproduced original analyses and conducted robustness checks of 110 articles that were published in leading economics and political science journals with mandatory data and code sharing policies17,18. We found that more than 85% of published claims were computationally reproducible. In robustness checks, our reanalyses showed that 72% of statistically significant estimates remain significant and in the same direction, and the median reproduced effect size is nearly the same as the originally published effect size (that is, 99% of the published effect size). Additionally, 6 independent research teams examined 12 pre-specified hypotheses about determinants of robustness. Research teams with more experience found lower levels of robustness, and robustness did not correlate with author characteristics or data availability.

DOI

10.1038/s41586-026-10251-x

Type

Journal article

Publication Date

2026-04-01T00:00:00+00:00

Volume

652

Pages

151 - 156

Total pages

5

Keywords

Reproducibility of Results, Politics, Research, Economics, Social Sciences, Science, Research Personnel

Permalink More information Close