6.050J | Spring 2008 | Undergraduate

Information and Entropy

Unit 8: Inference





The person most responsible for use of maximum entropy principles in various fields of science is Edwin T. Jaynes (July 5, 1922 – April 30, 1998).


Jaynes knew, of course, about Thomas Bayes and when on sabbatical in England sought out and photographed the Bayes tombstone.


  • The philosophy of assuming maximum uncertainty is discussed in Chapter 3 of Tribus, M. Thermostatics and Thermodynamics. Princeton, NJ: D. Van Nostrand Co, Inc., 1961.
  • Another good explanation, in terms of estimating probabilities of an unfair die is in Jaynes, E. T. “Information Theory and Statistical Mechanics (PDF).” In Statistical Physics. Brandeis Summer Institute 1962. New York, NY: W. A. Benjamin, Inc., 1963. pp. 181–218.
  • Buy at MIT Press Personal history by Jaynes, Edwin T. “Where Do We Stand on Maximum Entropy? (PDF - 2.5 MB)” In The Maximum Entropy Formalism. Edited by Raphael D. Levine and Myron Tribus. Cambridge, MA: MIT Press, 1978. pp. 15–118. ISBN: 9780262120807.
Learning Resource Types
Problem Sets with Solutions
Online Textbook
Programming Assignments