The Tool
Practical skills assignment
- Use a tool for making a dynamic executable document (R Markdown; Jupiter Notebook; Matlab Notebook) to write this week’s assignment. In your executable document, use your own data (or data from another paper) to make at least one reproducible figure.
- Zotero has partnered with Retraction Watch to automatically flag retracted papers in a user’s repository. Download Zotero if you do not already have it, and either import a repository from your current citation manager or manually import a few papers of your choice if you do not use a citation manager. Look through one of your repositories, and see if any papers are flagged as retracted. If not, add a retracted paper (you can find many on Retraction Watch’s database) to the repository. Is it correctly flagged as retracted? Do you think this flag is noticeable enough to be a useful tool in preventing the use of retracted papers?
In part 2 of your response paper, describe what you did in fulfilling this activity. What snags did you hit? What made this process easier or more difficult? Did you find any errors?
Useful Links and Resources
Dynamic documents: Sharing data and the code to make the figures all at once:
- R markdown intro.
- RMarkDown Tips and RMarkDown Lessons (particularly the first 9).
- DataColada: Eight things I do to make my open research more findable and understandable.
- Example of computational notebook accompanying paper, A guide to LIGO–Virgo detector noise and extraction of transient gravitational-wave signals, to reproduce all figures.
- Very elegant RMarkDown document to accompany Julia Leonard’s “Associations between cortical thickness and reasoning differ by socioeconomic status in development” paper. By downloading the data dictionary and this document, you can reproduce the whole paper, make all figures, etc.
- Here’s a beautiful example of using interactive visualizations of data and statistics.
- eLife Executable Research Articles.
- QRESP: The open-source software Qresp “Curation and Exploration of Reproducible Scientific Papers” facilitates the organization, annotation, and exploration of data presented in scientific papers.
Detecting and responding to mistakes and fraud:
- A proposal to replace peer review with “Red Teams.” An author gave ‘red teams’ a financial incentive to find errors in a submission-ready manuscript. Why? Part 1; What happened? Part 3.
- The Buck stops nowhere. Blog about being the “Data police.” James Heathers, 2017
- Evidence of Fraud in an Influential Field Experiment About Dishonesty - Aug 2021, DataColada.
- The fight against fake-paper factories that churn out sham science. Nature News Feature, 23 March 2021.
- https://retractionwatch.com/ (I found the FAQ interesting) & tool for searching the retraction watch database.
- DataColada: Reducing Fraud in Science.
- Notes from Computational Research Integrity Day 2021.
- Schneider, J., Woods, N. D., Proescholdt, R., Fu, Y., & the RISRS Team. 2021. “Recommendations from the Reducing the Inadvertent Spread of Retracted Science: Shaping a Research and Implementation Agenda Project.” MetaArXiv. July 29. doi:10.31222/osf.io/ms579.
- Serra-Garcia, M., & Gneezy, U. (2021). “Non-replicable publications are cited more than replicable ones.” Science Advances, 7(21), eabd1705.
- A bibliography of papers about retraction.
- Suelzer, E.M., Deal, J., Hanus, K.L. (2020). “Challenges in Identifying the Retracted Status of an Article."JAMA Netw Open. 2021;4(6):e2115648. doi:10.1001/jamanetworkopen.2021.15648
