6.803 | Spring 2019 | Undergraduate

The Human Intelligence Enterprise

Schedule of Activities

Ajemian detangles neural nets

Paper

None. However, as an aid to memory, we offer notes taken by Dylan during lecture. We hope that they add useful perspective to your own notes, and make no claim that our notes are especially authoritative.

Edit: Prof. Ajemian’s slides (PDF), however, are especially authoritative.

Assignment

[Note: If you discuss the assignment with another student—which we encourage—indicate whom you have talked with in your submitted composition. Of course your submitted composition must be written entirely by you.]

On a total of one side of one sheet of paper, using 10 pt type or larger, with standard interline spacing and margins, respond to all the following:

Much to your surprise, you have just received a call from your old friend Alyssa P. Hacker, now a high-ranking member of the Schwartzman College of Computing. Alyssa would like your help designing a bold new curriculum in which Brain and Cognitive Science students would all get basic training in machine learning and artificial neural nets. “After all,” she explains. “Artifical neural nets reveal much about human learning and the hierarchical cortical structures in the brain.” Having just heard Robert Ajemian speak, you have some doubts. However, teaching a computationally-grounded approach seems exactly right, and 6.803 luminaries (such as Minsky, Winston, and Marr) have given you a wealth of ideas about the right way forward. You readily agree to help out.

You decide to respond to Alyssa by letter, articulating the ideas you think the cognitive science curriculum of the future should include.

  1. Write a letter in response, explaining Ajemian’s view on the limits of artificial nets, and—
  2. Argue, instead, for a few key ideas from 6.803 readings you think everyone studying mental processes should know.

Naturally, you should focus your argument on conveying fundamental ideas in detail rather than, for example, minutiae about which courses should be taught and how.

If we are to capture human learning and cognition, we must envision the exotic architectures that make them possible.

  • New models of brain hardware can shed light on new models of brain software.
  • Design rule: Fit the approach to the problem you’re trying to solve.
  • The trendiest tools are not always the best tools for the job.
  • For example, modern computer systems are not the best metaphor for how brain hardware works. (6 key differences between computers and brains)
  • And artificial neural networks with backprop are not the best model of how brains learn. (6 unrealistic aspects of back prop)
    • In particular, backprop requires global lockstep coordination where brains are mostly asyncronous and local; and even recurrent neural networks pass information in one direction rather than many, as in the brain (compare recursive functions vs coroutines.)
  • And artificial neural networks with backprop are not the best engineering solution for every learning problem.
    • “Bulldozer computing” works well on object recognition, but fails on motor control (which has too-large state space) and medical diagnosis (which integrates many kinds of info.)
    • We should tailor new tools to such problems.
    • All of the recent ML performance improvements have come not from new ideas about learning, but rather from advances in computing power and data size.
  • Moore’s law will asymptote.
  • We must imagine new exotic architectures and learning mechanisms, tailored to the problems we’re trying to solve.
  • Surprise: There are other kinds of “neural networks” that are better brain models or better at solving ML problems:
    • Marr-Albus model of the cerebellum.
    • Hopfield networks
    • Self-organizing maps
    • Decision trees.

(C) 2019 Dylan Holmes. This work is not covered by MIT OpenCourseWare’s Creative Commons license but is licensed under the Creative Commons Attribution-NoDerivatives 4.0 International License. To view a copy of this license, visit https://creativecommons.org/licenses/by-nd/4.0/.

Course Info

As Taught In
Spring 2019
Learning Resource Types
Written Assignments