6.050J | Spring 2008 | Undergraduate

Information and Entropy

Unit 5: Probability

“Probability is nothing but common sense reduced to calculation.”
Pierre-Simon Laplace (1749–1827)


  • Notes, Chapter 5: Probability (PDF)
  • Huffmann, David A. “A Method for the Construction of Minimum-Redundancy Codes.” Proceedings of the Institute of Radio Engineers 40 (September 1952): 1098–1101.
  • Towser’s Wonderland Park greyhound handicaps, Boston Globe, February 27, 2005 (and results, Boston Globe, February 28, 2005)





General Technical Books

There are many excellent texts on probability, many of which do not assume a familiarity with mathematics beyond introductory calculus. Most books on communications include a summary of the necessary background in probability.

  • Drake, Alvin W. Fundamentals of Applied Probability Theory. New York, NY: McGraw-Hill, 1967. ISBN: 9780070178151.
    Prof. Drake taught 6.041 Probabilistic Systems Analysis for many years before he retired. He died Oct. 30, 2005. (Obituary)
  • Bertsekas, Dimitri P., and John N. Tsitsiklis. Introduction to Probability. Belmont, MA: Athena Scientific, 2002. ISBN: 9781886529403. Used in 6.041 today.
  • Applebaum, David. Probability and Information. New York, NY: Cambridge University Press, 2008. ISBN: 9780521727884.
    Chapter 4, Probability, contains a good comparison of the different philosophies underlying probability (symmetry, subjective, frequency).
  • Haykin, Simon. Communication Systems. 4th ed. New York, NY: John Wiley and Sons, Inc., 2000. ISBN: 9780471178699.
    Appendix 1, Probability Theory.
Learning Resource Types
Problem Sets with Solutions
Online Textbook
Programming Assignments