This section contains lecture notes and some associated readings.
Complete lecture notes (PDF - 7.7MB)
| TOPICS | LECTURE NOTES | READINGS |
|---|---|---|
|
The role of convexity in optimization Duality theory Algorithms and duality |
Lecture 1 (PDF - 1.2MB) | |
|
Convex sets and functions Epigraphs Closed convex functions Recognizing convex functions |
Lecture 2 (PDF) | Section 1.1 |
|
Differentiable convex functions Convex and affine hulls Caratheodory’s theorem |
Lecture 3 (PDF) | Sections 1.1, 1.2 |
|
Relative interior and closure Algebra of relative interiors and closures Continuity of convex functions Closures of functions |
Lecture 4 (PDF) | Section 1.3 |
|
Recession cones and lineality space Directions of recession of convex functions Local and global minima Existence of optimal solutions |
Lecture 5 (PDF - 1.0MB) | Sections 1.4, 3.1, 3.2 |
|
Nonemptiness of closed set intersections Existence of optimal solutions Preservation of closure under linear transformation Hyperplanes |
Lecture 6 (PDF - 1.4MB) | |
|
Review of hyperplane separation Nonvertical hyperplanes Convex conjugate functions Conjugacy theorem Examples |
Lecture 7 (PDF) | Sections 1.5, 1.6 |
|
Review of conjugate convex functions Min common / max crossing duality Weak duality Special cases |
Lecture 8 (PDF - 1.2MB) | Sections 1.6, 4.1, 4.2 |
|
Minimax problems and zero-sum games Min common / max crossing duality for minimax and zero-sum games Min common / max crossing duality theorems Strong duality conditions Existence of dual optimal solutions |
Lecture 9 (PDF) | Sections 3.4, 4.3, 4.4, 5.1 |
|
Min common / max crossing Theorem III Nonlinear Farkas’ lemma / linear constraints Linear programming duality Convex programming duality Optimality conditions |
Lecture 10 (PDF) | Sections 4.5, 5.1, 5.2, 5.3.1–5.3.2 |
|
Review of convex programming duality / counterexamples Fenchel duality Conic duality |
Lecture 11 (PDF) | Sections 5.3.1–5.3.6 |
|
Subgradients Fenchel inequality Sensitivity in constrained optimization Subdifferential calculus Optimality conditions |
Lecture 12 (PDF) | Section 5.4 |
|
Problem structure Conic programming |
Lecture 13 (PDF) | |
|
Conic programming Semidefinite programming Exact penalty functions Descent methods for convex optimization Steepest descent method |
Lecture 14 (PDF) | Chapter 6: Convex Optimization Algorithms (PDF) |
|
Subgradient methods Calculation of subgradients Convergence |
Lecture 15 (PDF) | |
|
Approximate subgradient methods Approximation methods Cutting plane methods |
Lecture 16 (PDF) | |
|
Review of cutting plane method Simplicial decomposition Duality between cutting plane and simplicial decomposition |
Lecture 17 (PDF) | |
|
Generalized polyhedral approximation methods Combined cutting plane and simplicial decomposition methods |
Lecture 18 (PDF) | Bertsekas, Dimitri, and Huizhen Yu. “A Unifying Polyhedral Approximation Framework for Convex Optimization.” SIAM Journal on Optimization 21, no. 1 (2011): 333–60. |
|
Proximal minimization algorithm Extensions |
Lecture 19 (PDF) | |
|
Proximal methods Review of proximal minimization Proximal cutting plane algorithm Bundle methods Augmented Lagrangian methods Dual proximal minimization algorithm |
Lecture 20 (PDF - 1.1MB) | |
|
Generalized forms of the proximal point algorithm Interior point methods Constrained optimization case: barrier method Conic programming cases |
Lecture 21 (PDF) | |
|
Incremental methods Review of large sum problems Review of incremental gradient and subgradient methods Combined incremental subgradient and proximal methods Convergence analysis Cyclic and randomized component selection |
Lecture 22 (PDF) |
Bertsekas, Dimitri. “Incremental Gradient, Subgradient, and Proximal Methods for Convex Optimization: A Survey.” (PDF) Laboratory for Information and Decision Systems Report LIDS-P-2848, MIT, August 2010.
|
|
Review of subgradient methods Application to differentiable problems: gradient projection Iteration complexity issues Complexity of gradient projection Projection method with extrapolation Optimal algorithms |
Lecture 23 (PDF) | |
|
Gradient proximal minimization method Nonquadratic proximal algorithms Entropy minimization algorithm Exponential augmented Lagrangian method Entropic descent algorithm |
Lecture 24 (PDF) |
Beck, Amir, and Marc Teboulle. “Gradient-Based Algorithms with Applications to Signal-Recovery Problems.” In Convex Optimization in Signal Processing and Communications. Edited by Daniel Palomar and Yonina Eldar. Cambridge University Press, 2010. ISBN: 9780521762229. Beck, Amir, and Marc Teboulle. “Mirror Descent and Nonlinear Projected Subgradient Methods for Convex Optimization.” Operations Research Letters 31, no. 3 (2003): 167–75. Bertsekas, Dimitri. Nonlinear Programming. Athena Scientific, 1999. ISBN: 9781886529007. |
|
Convex analysis and duality Convex optimization algorithms |
Lecture 25 (PDF - 2.0MB) |
