GIVE NOW about ocw help & faqs contact us
6.S890 | Fall 2024 | Graduate

# Topics in Multiagent Learning

### Part I: Normal-Form Games

#### Session 1: Introduction

Topics: Introduction to the course and logistics.

#### Session 2: Setting and Equilibria: The Nash Equilibrium

Topics: Definition of normal-form games. Solution concepts and Nash equilibrium. Nash equilibrium existence theorem.

#### Session 3: Setting and Equilibria: The Correlated Equilibrium

Topics: Topological and computational properties of the set of Nash equilibria in normal-form games. Connections with linear programming. Definition of correlated and coarse correlated equilibria; relationships with Nash equilibria.

#### Session 4: Learning in Games: Foundations

Key Date: Problem set 1 out.

Topics: Regret and hindsight rationality. Definition of regret minimization and relationships with equilibrium concepts.

#### Session 5: Learning in Games: Algorithms (part I)

Topics: General principles in the design of learning algorithms. Follow-the-leader, regret matching, multiplicative weights update, online mirror descent.

#### Session 6: Learning in Games: Algorithms (part II)

Topics: Optimistic mirror descent and optimistic follow-the-regularized-leader. Accelerated computation of approximate equilibria.

#### Session 7: Learning in Games: Bandit Feedback

Topics: From multiplicative weights to Exp3. General principles. Obtaining high-probability bounds.

#### Session 8: Learning in Games: Φ-Regret Minimization

Topics: Gordon, Greenwald, and Marks (2008); Blum and Mansour; Stolz-Lugosi.

### Part II: Extensive-Form Games

#### Session 9: Foundations of Extensive-Form Games

Topics: Complete versus imperfect information. Kuhn’s theorem. Normal-form and sequence-form strategies. Similarities and differences with normal-form games.

#### Session 10: Learning in Extensive-Form Games

Key Date: Problem set 2 out.

Topics: No-regret algorithms for extensive-form games. Counterfactual utilities and counterfactual regret minimization (CFR).

#### Session 11: Equilibrium Refinements

Topics: Sequential irrationality. Extensive-form perfect equilibria and quasi-perfect equilibrium.

#### Session 12: No class (student holiday)

#### Session 13: Project ideas and brainstorming

#### Session 14: Project break

Coincides with INFORMS 2024 Annual Meeting.

#### Session 15: Project break

Coincides with INFORMS 2024 Annual Meeting.

#### Session 16: Deep Reinforcement Learning for Large-Scale Games (part I)

Topics: Rough taxonomy of deep RL methods for games. Decision-time planning in imperfect-information games, construction of superhuman agents for no-limit hold’em poker. Public belief states techniques (ReBeL).

#### Session 17: Deep Reinforcement Learning for Large-Scale Games (part II)

Topics: PPO and magnetic mirror descent.

### Part III: Other Structured Games

#### Session 18: Combinatorial Games and Kernelized MWU (Part I)

Topics: Example of combinatorial games. Kernelized multiplicative weights update algorithm.

#### Session 19: Combinatorial Games and Kernelized MWU (Part II)

Topics: Example of combinatorial games. Kernelized multiplicative weights update algorithm.

#### Session 20: Computation of Exact Equilibria

Key Date: Problem set 3 out.

Topics: A second look at the minimax theorem. Hart and Schmeidler’s proof of existence of correlated equilibria. Ellipsoid against hope algorithm.

#### Session 21: Stochastic Games

Topics: Minimax theorem, and existence of equilibrium. Stationary Markov Nash equilibria. Coarse correlated and correlated equilibria in stochastic games.

### Part IV: Complexity of Equilibrium Computation

#### Session 22: PPAD-Completeness of Nash Equilibria (part I)

Topics: Sperner’s lemma. The PPAD complexity class. Nash ∈ PPAD.

#### Session 23: PPAD-Completeness of Nash Equilibria (part II)

Topics: Arithmetic circuit SAT. PPAD-hardness of Nash equilibria. Project break and presentations.

#### Session 24: Project break

#### Session 25: No class (Thanksgiving)

#### Session 26: Project presentations

#### Session 27: Project presentations

## Course Info

##### Instructor

- Prof. Gabriele Farina

##### Departments

- Electrical Engineering and Computer Science

##### As Taught In

Fall 2024

##### Level

Graduate

##### Topics

- Engineering
  - Computer Science
    - Artificial Intelligence

##### Learning Resource Types

notes Lecture Notes
assignment_turned_in Problem Set Solutions
assignment Problem Sets
group_work Projects
auto_stories Readings

Browse Resources

# You are leaving MIT OpenCourseWare

Please be advised that external sites may have terms and conditions, including license rights, that differ from ours. MIT OCW is not responsible for any content on third party sites, nor does a link suggest an endorsement of those sites and/or their content.