Tools for Robust Science

Syllabus

Meeting Times

1 session / week; 3 hours / session

Synchronous participation is required because the class involves substantial discussion.

Prerequisites

None. This course is intended for PhD and M-Eng students. Although based in Brain and Cognitive Science (course 9) and Computation and Cognition (course 6-9), students from other departments are welcome. Auditors at any career stage are welcome but expected to complete readings and assignments (including oral presentations, excluding the final project).

Description

We aspire to do science to make true discoveries, build cumulative knowledge through skeptical inquiry, and translate this knowledge into social goods like medicines, devices, or policies. Yet in reality, scientists have often failed to live up to these aspirations, because of bad habits and bad institutions. In the current moment, tremendous energy is being devoted across the cognitive and neuro-sciences to renewing our scientific practices. New tools are being developed, to improve credibility, facilitate collaboration, accelerate scientific discovery, and expedite translation of results. As institutions such as the NIH and EU research council are developing new policies requiring data management, accessibility and broader impacts, early career researchers must learn to fulfill these requirements.

Learning Outcomes

Students in this course will (i) identify obstacles to conducting robust scientific research, (ii) practice using current cutting-edge tools designed to overcome these obstacles by improving scientific practices and incentives, and (iii) critically evaluate these tools’ potential and limitations. Example tools we will investigate include shared pre-registration, experimental design, data management plans, meta-data standards, repositories, FAIR code, open source data processing pipelines, alternatives to scientific paper formats, alternative publishing agreements, citation audits, reformulated incentives for hiring and promotion, and more.

Structure of the Class

Each week we will consider one major type of challenge that we face as scientists, and some tools being developed to address that challenge. 

Preparation for each week’s class will include:

• Background reading or other materials (e.g. video lectures) that describe the scope of the challenge. 
• An activity or task to develop skills and practical experience with the relevant tools.
• The weekly preparation work should take 4-6 hours per week.
• Each meeting of the seminar will have some combination of four components:
  1. The first 30 min of each class session will be used for 1:1 meetings with course staff and peers. Meetings with course staff will be used to develop final projects. Peer meetings will be used for feedback on final projects, as well as for discussion of the response papers.
  2. Full group opening discussion of the week’s challenge, particularly including relevant personal experiences.
  3. Oral presentations by participants. 
  4. A discussion of our collective critical evaluation of the week’s tool, including opportunities for improvement.

Required Work

This is a graduate class. Grades are based on making an authentic genuine effort and contributing to our joint intellectual development. We do this work well because we care about it, and out of respect for one another. A good faith effort on each aspect of the required work will get full marks. Student grades will be based on four kinds of required work:

1. Response papers (30pts). Every student every week, due the previous Friday at end-of-day. Three sections. Total length approximately 1 page, single spaced. In your view, what is the hardest part of this week’s challenge? Include any relevant personal experiences you’ve had. What did you accomplish in the practical activity? How do you critically evaluate the potential of this tool to address this challenge? How will it help scientists/science/the community, and what is still missing? 3 pts per paper (1 for each section). 0 pts for late work. The maximum total points you can earn is 30 (3*10weeks), scored out of 30.
2. In-class discussions (20 pts). 3pts per week. Must be synchronous. Includes contributions to the discussion, giving oral presentations, and asking questions after others’ oral presentations. As above, up to two weeks can be missed with no penalty.
3. Oral presentations (20pts). Participants will give oral presentations of the same material and structure as the written response papers, but in more depth. The number of presentations per class, and the number of presentations per participant, will be determined once we know how many people are in the seminar.
4. Final projects (30pts). Students taking the class for credit will do a final project. There are many options for the format of this project, including an essay on a topic related to the class, or a project developing or applying one of the tools discussed in the class.

Ethics of the Class 

We value mutual respect and trust between teachers and students. I enjoy working hard to teach students who are working hard to learn. I promise to do my best to make the classes engaging and to adapt to your feedback. I expect you to arrive on time, so you don’t disrupt the class, and to devote genuine intellectual energy to the material. This is a discussion-heavy class, and we will sometimes discuss ideas that are controversial. In order to have meaningful dialogue it will therefore be important to do all that we can to make everyone in the class feel comfortable in these discussions. On the first day of class, we will collaboratively generate ground rules for discussion (e.g., ‘respect others’ lived experiences’). Working together or helping one another on the weekly assignments is definitely encouraged. However, each student needs to produce their own output for the week. For example, if you are tasked to draft a pre-registration, each student should create a unique document. All submitted written work must be 100% your own original writing. Copying sentences or ideas from other students or from any other source is plagiarism. Plagiarism is unfair (to the other students) and disrespectful (to the intellectual challenge of the course), and I take it seriously. For questions, feedback, or issues that come up that are specific to you, please email the instructors.

Accommodations

We are committed to creating a community where everyone feels supported. If you have special needs or requests for the course, share these with us on the intake form. If these needs change throughout the term, contact us.

Note that the information shared here was intended for residential students and is included to provide insight into how the course was conducted at MIT. OCW is not a distance education program, we do not offer instructor interactions, grading, or certification of completion.