LEC #  TOPICS  KEY DATES 

1  Introduction, entropy  
2  Jensen’s inequality, data processing theorem, Fanos’s inequality  
3  Different types of convergence, asymptotic equipartition property (AEP), typical set, joint typicality  
4  Entropies of stochastic processes  Problem set 1 due 
5  Data compression, Kraft inequality, optimal codes  Problem set 2 due 
6  Huffman codes  Problem set 3 due 
7  ShannonFanoElias codes, SlepianWolf  
8  Channel capacity, binary symmetric and erasure channels  
9  Maximizing capacity, BlahutArimoto  Problem set 4 due 
10  The channel coding theorem  
11  Strong coding theorem, types of errors  Problem set 5 due 
12  Strong coding theorem, error exponents  
Inclass midterm  
13  Fano’s inequality and the converse to the coding theorem  Problem set 6 due 
14  Feedback capacity  
15  Joint source channel coding  Problem set 7 due 
16  Differential entropy, maximizing entropy  
17  Additive Gaussian noise channel  Problem set 8 due 
18  Gaussian channels: parallel, colored noise, intersymbol interference  
19  Gaussian channels with feedback  Problem set 9 due 
20  Multiple access channels  
21  Broadcast channels  Problem set 10 due 
Inclass presentations (2 sessions)  
22  Finite state Markov channels  
23  Channel side information, wideband channels 
Calendar
Instructor:  
Course Number: 

Departments:  
As Taught In:  Spring 2010 
Level: 
Graduate

Topics
Learning Resource Types
assignment
Problem Sets
notes
Lecture Notes