| 1 |
Introduction, entropy |
(PDF) |
| 2 |
Jensen's inequality, data processing theorem, Fanos's inequality |
(PDF) |
| 3 |
Different types of convergence, asymptotic equipartition property (AEP), typical set, joint typicality |
(PDF) |
| 4 |
Entropies of stochastic processes |
(PDF) |
| 5 |
Data compression, Kraft inequality, optimal codes |
(PDF) |
| 6 |
Huffman codes |
(PDF) |
| 7 |
Shannon-Fano-Elias codes, Slepian-Wolf |
(PDF 1) (PDF 2) |
| 8 |
Channel capacity, binary symmetric and erasure channels |
(PDF) |
| 9 |
Maximizing capacity, Blahut-Arimoto |
(PDF) |
| 10 |
The channel coding theorem |
(PDF) |
| 11 |
Strong coding theorem, types of errors |
(PDF) |
| 12 |
Strong coding theorem, error exponents |
(PDF) |
| 13 |
Fano's inequality and the converse to the coding theorem |
(PDF) |
| 14 |
Feedback capacity |
(PDF) |
| 15 |
Joint source channel coding |
(PDF) |
| 16 |
Differential entropy, maximizing entropy |
(PDF) |
| 17 |
Additive Gaussian noise channel |
(PDF) |
| 18 |
Gaussian channels: parallel, colored noise, inter-symbol interference |
(PDF) |
| 19 |
Gaussian channels with feedback |
(PDF) |
| 20 |
Multiple access channels |
(PDF) |
| 21 |
Broadcast channels |
(PDF) |
| 22 |
Finite state Markov channels |
(PDF) |
| 23 |
Channel side information, wide-band channels |
(PDF) |
