Course Format
Lecture: 1 session/week, 2.5 hours/session
This course includes in-class coding work and group project components.
Course Description
HST.953/6.8850 is a course about the practical considerations for operationalizing machine learning in healthcare settings. Artificial intelligence (AI) has the potential to transform healthcare worldwide, bearing promises of increased accuracy, efficiency, and cost-effectiveness, in areas as diverse as drug discovery, clinical diagnosis, and disease management. Furthermore, AI has been promoted as a tool that could expand the reach of quality healthcare to traditionally underserved patients and regions. But even with appropriate representation of marginalized communities with high quality data, the social patterning of the data generation process can still produce AI that is bound to preserve and even scale existing disparities in care with resulting inequities in patient outcomes. Creating algorithms from the digital exhaust of flawed human systems, by AI developers who are not cognizant of the backstory of the data, risks cementing inequities as permanent fixtures in healthcare delivery systems.
This course will introduce students to a portfolio of methodologies that learn patterns from the data. More importantly, it will explore data issues which if not addressed will have profound consequences on downstream prediction, classification, and optimization tasks.
The course will involve three homework assignments (one each on machine learning, visualization, and implementation) followed by a course project proposal and presentation. All students are required to complete human subjects training and submit proof of access for MIMIC-III and the eICU-CRD databases. All students regardless of their enrollment status are expected to join a project group and contribute to a final project.
Prerequisites
HST.953 is not intended to teach graduate machine learning or visualization skills to students, and we expect that students will have some working knowledge of both in order to complete homework assignments and the project. We recommend the following courses, or some equivalent experience with subject matter in ML, visualization, and HCI:
- 6.867 Machine Learning
- 6.S897 / HST.956 Machine Learning for Healthcare
- 6.813/6.831 User Interface Design & Implementation
Learning Objectives
Upon successful completion of this course, you should be able to do the following:
- Work with data scientists, social scientists and clinicians across the life cycle of health AI and apply systems thinking to the application of AI to healthcare
- Learn good code documentation for reproducibility of AI development
- Develop a critical understanding of how the dataset came about from collection to aggregation to standardization
- Perform exploratory data analysis with a special emphasis on data bias
- Understand basic principles of different machine learning methodologies
- Interpret and communicate analysis results
- Think about potential downstream harm from algorithm implementation
We will start with a primer on machine learning concepts including but not limited to cross-validation, data leakage, benchmarks, performance metrics, and fairness evaluation. Publicly available high-resolution datasets (not registries) will be leveraged.
Other Semester Activities
1. The Bias-athon is designed to address and mitigate biases in artificial intelligence (AI) systems. This workshop will leverage interdisciplinarity to identify, understand, and develop strategies to understand biases in clinical AI datasets. Participants will engage in hands-on sessions where they explore various types of biases, such as measurement bias, and variation in the degree of monitoring from social determinants of care, and their impact on AI performance.
2. A prompt-athon and red teaming will focus on enhancing the effectiveness and reducing the bias of large language models. This workshop is designed for clinicians who are already or who are thinking of using these tools for summarizing patient course, drafting content for progress notes and letters to other providers and to the patients, and soliciting differential diagnoses, treatment recommendations, and prognostication. Participants will be introduced to various prompt engineering techniques that can leverage the power of this technology. Through collaborative exercises, attendees will experiment with different types of prompts, analyze the outputs, and refine their strategies to achieve better results. The event will also include discussions on the challenges of prompt design, such as avoiding ambiguity and ensuring context-appropriateness.
3. The Health AI Systems Thinking for Equity (HASTE) Policy Workshop is organized to explore the regulatory and ethical frameworks surrounding the use of AI technologies. Sessions will cover a range of topics, including transparency and accountability, power structures, and the political economy that drives the impact of AI. Participants will engage in brainstorming and dialogue and propose solutions to complex policy issues. The goal is to engender a systems thinking mindset among developers and users of AI to improve population health.
Course Structure
Classroom Participation
This course includes in-class coding work and group project components.
Class Attendance and Participation
Class attendance is mandatory. Students will be excused from class in the event of a family emergency, medical issue, religious observance, or other extenuating circumstance, and should contact the instructors or TA beforehand.
Inclusivity Statement
Diversity and inclusiveness are fundamental to public health education and practice. Students are encouraged to have an open mind and respect differences of all kinds. We share responsibility with you for creating a learning climate that is hospitable to all perspectives and cultures; please contact the instructors if you have any concerns or suggestions.
Bias-Related Incident Reporting
MIT believes all members of our community should be able to study and work in an environment where they feel safe and respected. As a mechanism to promote an inclusive community, we have created an anonymous bias-related incident reporting system.
Academic Integrity
Each student in this course is expected to abide by MIT’s standards of academic integrity. All work submitted to meet course requirements is expected to be a student’s own work. In the preparation of work submitted to meet course requirements, students should always take great care to distinguish their own ideas and knowledge from information derived from sources.
Students must acknowledge any collaboration and its extent in all submitted work. This requirement applies to collaboration on editing as well as collaboration on substance.
Accommodations for Students with Disabilities
MIT provides academic accommodations to students with disabilities. Any requests for academic accommodations should ideally be made before the first week of the semester, except for unusual circumstances, so arrangements can be made.
Absence Due to Religious Holidays
According to Chapter 151c, Section 2B, of the General Laws of Massachusetts, any student in an educational or vocational training institution, other than a religious or denominational training institution, who is unable, because of his or her religious beliefs, to attend classes or to participate in any examination, study, or work requirement on a particular day shall be excused from any such examination or requirement which he or she may have missed because of such absence on any particular day, provided that such makeup examination or work shall not create an unreasonable burden upon the School.
Course Evaluations
Constructive feedback from students is a valuable resource for improving teaching. The feedback should be specific, focused, and respectful. It should also address aspects of the course and teaching that are positive as well as those which need improvement.
Completion of the evaluation is a requirement for each course. Your grade will not be available until you submit the evaluation. In addition, registration for future terms will be blocked until you have completed evaluations for courses in prior terms.
Grading
| Activities | Percentages |
|---|---|
| Weekly Reflections: The weekly reflections, corresponding to weeks 2 through 10, will be done as a Canvas discussion, are due before class, and are worth 1 point (1.67% of your grade) per week. This means that reflections are worth a total of 15% of your grade. | 15% |
| Three Problem Sets: Problem sets 1, 2, and 3 are each worth 10 points, or 16.67% of your grade. This means that problem sets are worth a total of 50% of your grade. | 50% |
| Course Final Project: The submission of the project teams is worth 1 point (1.67% of your grade), the final project presentation is worth 10 points (16.67% of your grade), and the final project write-up is worth 10 points (16.67% of your grade). |
1.67% 16.67% 16.67% |
Plagiarism: Student code submissions may be submitted by the instructors to a plagiarism detection tool for a review of similarity and detection of possible plagiarism. Submissions will be used solely for the purpose of detecting similarity, and are not retained indefinitely on the server; typically results are deleted after 14 days but may be removed sooner. For more information on the tool used, refer to https://theory.stanford.edu/~aiken/moss/.
Projects and Authorship
A note on collaboration: Research is a collaborative activity, and we encourage all students to collaborate and learn from each other. In general, when you put your name on something for research, you must: a) have materially contributed to the work, b) be able to defend the research, and c) acknowledge the contribution of others. Keep this in mind when working together and submitting material for evaluation.
A note on authorship: As noted, the expectation is that by the end of the course the final project will be sufficiently developed to submit to a peer-reviewed journal. The author order can be a somewhat controversial issue and is left to the project participants to decide. We would strongly encourage you to discuss what the order will be, or what philosophy you will use to decide the order while forming groups. In the case of a dispute during or after the course, the instructors will likely not be able to mediate in any meaningful way. We would also recommend equal authors (now more common), but the decision is left to each team.
For the clinicians: If you expect a certain level of authorship (first, last, etc.) you should mention this in your project pitch. Keep in mind that this is a two-way street involving both clinicians and data scientists. If a project fails to garner enough interest, it may not be able to be completed as part of the course.
A note on acknowledgement: Papers that result from work done during this course should recognize the contributions of the course in an acknowledgement or in other sections. The suggested language is as follows: “This manuscript was composed by participants in the HST.953 course at the Massachusetts Institute of Technology, Fall 2024.”
