6.441 | Spring 2010 | Graduate

Information Theory


Course Meeting Times

Lectures: 2 sessions / week, 1.5 hours / session

Course Description

6.441 offers an introduction to the quantitative theory of information and its applications to reliable, efficient communication systems. Topics include mathematical definition and properties of information, source coding theorem, lossless compression of data, optimal lossless coding, noisy communication channels, channel coding theorem, the source channel separation theorem, multiple access channels, broadcast channels, Gaussian noise, and time-varying channels.

Reading Materials


Cover, Thomas, and Joy Thomas. Elements of Information Theory. 2nd ed. New York, NY: Wiley-Interscience, 2006. ISBN: 9780471241959.

Gallagher, Robert. Information Theory and Reliable Communication. New York, NY: Wiley, 1968. ISBN: 9780471290483.


Problem Sets

There will 10 problem sets. You may collaborate on problem sets, but you must indicate for each problem the name(s) of your collaborator(s).


There will be a project based on one of the extra papers. The choice of papers will be up to the group. The project may be done by groups of 1-4 people. The project will require a project presentation of 12 minutes per person (the presentation time per project will be proportional to the number of people in the group). There will be a written report of no more of 3 pages per person.

Main dates:

Lec #11: Choice of paper due
Before Lec #13: Each group meets with instructor to discuss the project
Before Lec #22: Presentations in class and at a scheduled time outside class (optional attendance)


Only the reading assignments out of the book are required. All material not in the book will be covered in class.

The supplementary readings are optional, except for the paper that you select as part of your project. The supplementary readings are coded for difficulty. One star: Accessible. Two stars: Requires significant mathematical maturity. Three stars: Expert level.

Grading Policy

Midterm 25%
Final 45%
Homework 10%
Project 20%

Course Info

As Taught In
Spring 2010