Special software is required to use some of the files in this section: .ps. This section contains documents that could not be made accessible to screen reader software. A "#" symbol is used to denote such documents.

When you click the Amazon logo to the left of any citation and purchase the book (or other media) from Amazon.com, MIT OpenCourseWare will receive up to 10% of this purchase and any other purchases you make during that visit. This will not increase the cost of your purchase. Links provided are to the US Amazon site, but you can also support OCW through Amazon sites in other regions. Learn more.

Readings

• Notes, Chapter 9: Principle of Maximum Entropy (PDF)

Assignments

• Problem Set 8 (PDF)
• Problem Set 8 Solutions (PDF)
• More Quiz Review

Resources

Technical

The person most responsible for use of maximum entropy principles in various fields of science is Edwin T. Jaynes. The seminal papers are:

• Jaynes, E. T. "Information Theory and Statistical Mechanics." Physical Review 106 (May 15, 1957): 620-630. (PDF - 2.1 MB)^# (PS - 2.5 MB)
  This paper started the use of the Principle of Maximum Entropy in physics.
• ———. "Information Theory and Statistical Mechanics II." Physical Review 108 (October 15, 1957): 171-190. (PDF - 3.6 MB)^# (PS - 4.5 MB)
  Continuation of the previous reference.

Historical

• Joseph-Louis Lagrange (1736 - 1813) biography
• Edwin T. Jaynes (1922 - 1998) biography, photograph, bibliography

Books

• The philosophy of assuming maximum uncertainty is discussed in Chapter 3 of Tribus, M. Thermostatics and Thermodynamics. Princeton, NJ: D. Van Nostrand Co, Inc., 1961.
• Another good explanation, in terms of estimating probabilities of an unfair die is in Jaynes, E. T. "Information Theory and Statistical Mechanics." In Statistical Physics. Brandeis Summer Institute 1962. New York, NY: W. A. Benjamin, Inc., 1963. pp. 181-218. (PDF)^# (PS - 1.1 MB)
• Personal history by Jaynes, Edwin T. "Where Do We Stand on Maximum Entropy?" In The Maximum Entropy Formalism. Edited by Raphael D. Levine and Myron Tribus. Cambridge, MA: MIT Press, 1978. pp. 15-118. ISBN: 9780262120807. (PDF - 2.5 MB)^# (PS - 3.2 MB)

UNITS: Bits and Codes | Compression | Noise and Errors | Probability | Communications | Processes | Inference | Maximum Entropy | Physical Systems | Energy | Temperature | Quantum Information