9.401 | Fall 2022 | Graduate

Tools for Robust Science

Week 3 Challenge: Experiments Cannot Be Reproduced by Other Labs

The Challenge

A defining feature of scientific knowledge is that it is replicable: another scientist should be able to repeat the procedure and re-generate the same results. Yet in reality, there are many obstacles to replication. The first obstacle is often: published procedures are not described sufficiently, to allow another scientist to reproduce them. Other obstacles to replication will be discussed in later classes (e.g. experimenter degrees of freedom and publication bias mean that published results offer an inflated estimate of effect sizes; professional incentives for scientists often don’t reward reproducing or being reproducible). Today we focus on the challenge of how to make scientific procedures reproducible.


Everyone should read the resources under ’everyone.’ After that please read the additional readings most relevant to you.

Biological Sciences

Errington, T. M. (2019). “Reproducibility Project: Cancer Biology, Barriers to Replicability in the Process of Research.” Talk at the MetaScience Conference. Read the slides or watch the talk.

Systems Neuroscience

Banga, K., Benson, J., Bonacchi, N., et al. (2022). “Reproducibility of in-vivo electrophysiological measurements in mice.” bioRxiv.

Computational Science
Developmental Science
Social Science

Moody, J. W., Keister, L. A., & Ramos, M. C. (2022). “Reproducibility in the Social Sciences.” Annual Review of Sociology, 48.

Course Info

As Taught In
Fall 2022
Learning Resource Types
Written Assignments