6.050J | Spring 2008 | Undergraduate

Information and Entropy


Course Meeting Times

2 sessions / week, 2 hours / session


Physics I: Classical Mechanics (Offered in various forms at MIT as 8.01SC Classical Mechanics, 8.012 Physics I: Classical Mechanics, 8.01L Physics I: Classical Mechanics, 8.01T, or advanced standing credit)

Course Description

This subject is designed for MIT freshmen. Academic credit of 9 units (less than that given by a typical MIT subject) is provided.

Spring 2008 is the sixth offering of this subject. It was offered in Spring 2003, 2004, 2005, 2006, 2007 and, before then, three times while being developed, under another number, in Spring 2000, 2001, and 2002.

This subject is offered jointly by the Department of Electrical Engineering and Computer Science and the Department of Mechanical Engineering. Students may sign up for either 2.110J or 6.050J.


The course is taught from a set of notes written by Prof. Penfield, developed over several years of teaching this course. These readings are linked from each unit. Each week these readings are also supplemented by a plethora of online and print resources. Information and Entropy full course notes (PDF - 4MB)


Problem sets are not distributed. Instead, homework problems are described and assigned in class. In some cases, solutions are provided after the problems are due. Late submissions do not receive any credit. Problem sets from 2003 have been republished as part of each unit, to provide representative examples of the homework problems.



Closed book except that one sheet of 8 1/2 x 11 inch paper, with notes on both sides, is allowed. Coverage through Unit 8.

Final Examination

Closed book except that two sheets of 8 1/2 x 11 inch paper, with notes on both sides, are allowed.


Students are expected to access resources on the World Wide Web, both from the course site and elsewhere.


6.050J / 2.110J is a nine-unit subject. It is intended that the overall work required be approximately nine hours per week, including four hours of lecture and recitation. Any students who find themselves spending substantially more than nine hours any week should question whether they are stuck and might make more rapid progress if they asked the instructing staff for some hints or got advice from fellow students. In particular, students should avoid spending nonproductive time on the computer, either polishing a MATLAB exercise unnecessarily, or surfing the Web aimlessly.


Weak collaboration is permitted on problem sets. In this context the term “weak collaboration” means that two or more students may discuss the problems and their ways of approaching them, but each student must fully work out the problem and present only his or her own solution. Advice can be given and received, but no part of the solution can be copied from another, nor can identical portions appear in the submissions of two or more students. Any weak collaboration must be fully disclosed as part of the problem solution, for example by a phrase like “Alice Alison and Bob Robertson collaborated in part (b) by discussions of general approach.” Since weak collaboration involves discussions among two or more people, all must have compatible statements.

Help from people not taking this course is also permitted, provided that it is fully disclosed, and that the solution submitted was written in the privacy of the submitter’s own mind and body.

Strong collaboration is not permitted on problem sets. In this context the term “strong collaboration” is any collaboration in which work done by others is incorporated, with or without disclosure. Strong collaboration is normal and desirable in the work environment, where the principal purpose is to accomplish, as a team, some objective. In an academic setting, however, the purpose is to facilitate learning by each individual student, and strong collaboration does not support that goal.

It is, of course, a serious academic offense for a student to present another’s work as his or her own. It is also an offense to fail to report collaboration in accordance with course policy. Such offenses will be treated seriously.


Grades will be based on:

Participation in class 15%
Problem set solutions 20%
Mid-term quiz 20%
Final examination 30%
Subjective judgment of the instructing staff 15%


week # TOPICS
1 Units 1 & 2: Bits and codes
2 Unit 3: Compression
3 Unit 4: Noise and errors
4 Unit 5: Probability
5 Unit 6: Communications
6 Unit 7: Processes
7 Unit 8: Inference
8 Unit 9: Maximum entropy
9 Quiz
10 Unit 10: Physical systems
11 Unit 11: Energy
12 Unit 12: Temperature
13 Unit 13: Quantum information
14 Final exam
Learning Resource Types
Problem Sets with Solutions
Online Textbook
Programming Assignments