Information and Entropy

Unit 6: Communications

Readings

• Notes, Chapter 6: Communications (PDF)
• Bhatti, Saleem N. Channel Capacity (archived)
• ISO/OSI Network Model (archived), originally hosted by Indiana University
• Hedrick, Charles L. Introduction to the Internet Protocols. 1987
  • What is TCP/IP?
  • General description of the TCP/IP protocols

Assignments

• Problem Set 5 (PDF)
• Problem Set 5 Solutions (PDF)

Resources

Technical

• Shannon, Claude E. “A Mathematical Theory of Communication (PDF).” Bell System Technical Journal 27 (July and October 1948): 379–423 (Part I), 623–656 (Part II).
  These seminal papers are available in several forms.
  • Shannon, Claude E., and Warren Weaver. The Mathematical Theory of Communication. Urbana, IL: University of Illinois Press, 1998. ISBN: 9780252725463. (Incorporating a number of modifications and corrections by Shannon.)
  • Shannon, Claude E. “A Mathematical Theory of Communication.” 50th Anniversary Edition, printed for the 1998 IEEE International Symposium on Information Theory, MIT, Cambridge, UK: Cambridge University Press. August 16–21, 1998. (Based on 1949 book, with corrections.)
  • Reprinted in Key Papers in the Development of Information Theory. Edited by D. Slepian. New York, NY: IEEE Press, 1974.
  • Reprinted in Claude Elwood Shannon: Collected Papers. Edited by N. J. A. Sloane and A. D. Wyner. New York, NY: IEEE Press, 1993.
• Bhatti, Saleem N. Channel Capacity (archived). One of several excellent succinct expositions of channel capacity for symmetric binary channels.
• ISO/OSI Network Model (archived)
• IP version 6 and related specifications

Historical

• Claude E. Shannon (1916–2001). Obituary. MIT News, February 28, 2001.

General Technical Books

There are many excellent texts on communications, most of which assume a familiarity with mathematics beyond introductory calculus. Almost all cover Shannon’s work, and some also discuss feedback error control techniques.

• Cover, Thomas M., and Joy A. Thomas. Elements of Information Theory. New York, NY: John Wiley & Sons, Inc., 2006. ISBN: 9780471241959.
  Aimed at university seniors and first-year graduate students. One of several excellent books of that era. Professor Cover, at Stanford University, is one of the leaders in Information Theory.
• Hambley, Allan R. An Introduction to Communication Systems. New York, NY: W. H. Freeman, 1989. ISBN: 9780716781844.
  Discussion of various types of error control coding, including FEC (Forward Error Correction) and ARQ (Automatic Repeat Query) techniques, pp. 427 - 479.
• Gallager, Robert G. Information Theory and Reliable Communications. New York, NY: John Wiley & Sons, Inc., 1968. ISBN: 9780471290483.
  One of the early textbooks, designed for first-year graduate students, by one of the pioneers in communications, an MIT faculty member, later awarded the IEEE Medal of Honor, its most prestigious award.
• Haykin, Simon. Communication Systems. 4th ed. New York, NY: John Wiley & Sons, Inc., 2000. ISBN: 9780471178699.
• Pierce, John R. An Introduction to Information Theory: Symbols, Signals, and Noise. 2nd ed. New York, NY: Dover Publications, Inc., 1980. ISBN: 9780486240619.
  Mostly nonmathematical, by one of the nation’s great scientific contributors at AT&T Bell Laboratories, who was also interested in reaching a general audience. He was later on the faculty at Caltech. One of his interesting sideline activities was writing science fiction stories under the pen name J. J. Coupling. He died April 2, 2002 at the age of 92.