The Challenge: When experimenters can choose which data points and analyses to report, based on which produces the “best result” (called experimenter degrees of freedom), the literature gets filled with inflated and overconfident effect sizes. Describe your experience and perspective on this challenge. Have you encountered an example of this problem? What makes it particularly hard in your area of science?
The Tool: Pre-registered analysis pipeline. Describe what you did, in fulfilling the practical activity:
- Finding a pre-registered paper in your field, including where the pre-registration was stored, how easy it was to find from the paper, if the authors followed their pre-registration, and if they made clear where they deviated.
- Spending 30 minutes working on an outline/draft of a pre-registration of your own
- Identify a journal in your field that accepts submissions of Registered Reports (peer-reviewed pre-registrations)
Critical evaluation of the tool. What is the promise of this tool in addressing this challenge? What are the biggest obstacles?
This response paper should be about 1-page long, single-spaced. Total points: 3.
