Replication Games: Advancing Reproducibility with Open and Restricted Access Data

Reproducibility is often taught as a set of principles: share data, document code, pre-specify analyses, and make results transparent. Replication Games turn these principles into practice. In these hackathon-style events, researchers work in small teams to reproduce and stress-test published studies, learning not only whether results can be recreated, but also how empirical claims depend on documentation, coding choices, and analytical judgment. At the Institute for Replication, these events have become a scalable model for combining training, community-building, and cumulative science: teams develop practical skills while contributing structured evidence on the reproducibility and robustness of the literature.

In this talk, Replication Games will be discussed as both pedagogy and research infrastructure. We will also reflect on the recent CBS Replication Games in the Netherlands, where researchers attempted reproductions using restricted-access Dutch administrative microdata in a secure environment. That experience highlights a central frontier for reproducible science: how to make independent verification possible when the most policy-relevant data cannot be openly shared.