The Tool
Practical skills assignment
- Find a paper in your field that was pre-registered. Where was the pre-registration stored? Was it easy to find, from the paper? Compare the pre-registered analyses to the analyses reported in the paper. Did the authors follow their pre-registration? If not, did they make clear where they deviated?
- For one of your projects (at any stage from planning to post-publication), find a pre-registration template, or a pre-registration of a reasonably similar study. Spend 30 minutes on an outline/draft of a pre-registration. You don’t need to complete the pre-registration, but the idea is to learn about the challenges of writing a pre-registration.
- Identify a journal in your field that accepts submissions of Registered Reports (i.e. peer-reviewed pre-registrations)
In your response paper, describe what you accomplished in this task, including anything that was easier or harder than you expected.
Then, critically evaluate pre-registration of analysis plans as a tool to address experimenter degrees of freedom, and inflated and overconfident effect sizes in the scientific literature.
In part 2 of your response paper describe what you did, in fulfilling the practical activity as outlined above.
Useful Links and Resources
- How to pre-register: a Practical Guide
- A guide to registered reports
- Wieschowski, S., Laser, H., Sena, E. S., et al. (2020). “Attitudes towards animal study registries and their characteristics: An online survey of three cohorts of animal researchers.” PloS one, 15(1), e0226443.
Pre-registration:
- AsPredicted.org
- ClinicalTrials.gov
- https://www.socialscienceregistry.org/
- http://osf.io/
- https://egap.org/registry/
- http://ridie.3ieimpact.org/
- Getting started with DeclareDesign
- Preregistration information for NeurIPS.
- Journals that offer visible badges for pre-registration.
- What should a preregistration contain?
More arguments in favour of preregistration:
- https://www.phdontrack.net/open-science/preregistration/
- https://help.osf.io/article/145-preregistration
- A thoughtful discussion about pre-registration in mental health research by the editor of Biological Psychiatry, Deanna Barch.
- Many examples of pre-registration templates are linked in this thread from Twitter.
- Another example.
- A recent talk about preregistration.
Registered reports:
