TOPICS | LECTURE NOTES | READINGS |
---|---|---|

The role of convexity in optimization Duality theory Algorithms and duality |
Lecture 1 (PDF - 1.2MB) | |

Convex sets and functions Epigraphs Closed convex functions Recognizing convex functions |
Lecture 2 (PDF) | Section 1.1 |

Differentiable convex functions Convex and affine hulls Caratheodory's theorem |
Lecture 3 (PDF) | Sections 1.1, 1.2 |

Relative interior and closure Algebra of relative interiors and closures Continuity of convex functions Closures of functions |
Lecture 4 (PDF) | Section 1.3 |

Recession cones and lineality space Directions of recession of convex functions Local and global minima Existence of optimal solutions |
Lecture 5 (PDF - 1.0MB) | Sections 1.4, 3.1, 3.2 |

Nonemptiness of closed set intersections Existence of optimal solutions Preservation of closure under linear transformation Hyperplanes |
Lecture 6 (PDF - 1.4MB) | |

Review of hyperplane separation Nonvertical hyperplanes Convex conjugate functions Conjugacy theorem Examples |
Lecture 7 (PDF) | Sections 1.5, 1.6 |

Review of conjugate convex functions Min common / max crossing duality Weak duality Special cases |
Lecture 8 (PDF - 1.2MB) | Sections 1.6, 4.1, 4.2 |

Minimax problems and zero-sum games Min common / max crossing duality for minimax and zero-sum games Min common / max crossing duality theorems Strong duality conditions Existence of dual optimal solutions |
Lecture 9 (PDF) | Sections 3.4, 4.3, 4.4, 5.1 |

Min common / max crossing Theorem III Nonlinear Farkas' lemma / linear constraints Linear programming duality Convex programming duality Optimality conditions |
Lecture 10 (PDF) | Sections 4.5, 5.1, 5.2, 5.3.1–5.3.2 |

Review of convex programming duality / counterexamples Fenchel duality Conic duality |
Lecture 11 (PDF) | Sections 5.3.1–5.3.6 |

Subgradients Fenchel inequality Sensitivity in constrained optimization Subdifferential calculus Optimality conditions |
Lecture 12 (PDF) | Section 5.4 |

Problem structure Conic programming |
Lecture 13 (PDF) | |

Conic programming Semidefinite programming Exact penalty functions Descent methods for convex optimization Steepest descent method |
Lecture 14 (PDF) | Chapter 6: Convex Optimization Algorithms (PDF) |

Subgradient methods Calculation of subgradients Convergence |
Lecture 15 (PDF) | |

Approximate subgradient methods Approximation methods Cutting plane methods |
Lecture 16 (PDF) | |

Review of cutting plane method Simplicial decomposition Duality between cutting plane and simplicial decomposition |
Lecture 17 (PDF) | |

Generalized polyhedral approximation methods Combined cutting plane and simplicial decomposition methods |
Lecture 18 (PDF) | Bertsekas, Dimitri, and Huizhen Yu. "A Unifying Polyhedral Approximation Framework for Convex Optimization." SIAM Journal on Optimization 21, no. 1 (2011): 333–60. |

Proximal minimization algorithm Extensions |
Lecture 19 (PDF) | |

Proximal methods Review of proximal minimization Proximal cutting plane algorithm Bundle methods Augmented Lagrangian methods Dual proximal minimization algorithm |
Lecture 20 (PDF - 1.1MB) | |

Generalized forms of the proximal point algorithm Interior point methods Constrained optimization case: barrier method Conic programming cases |
Lecture 21 (PDF) | |

Incremental methods Review of large sum problems Review of incremental gradient and subgradient methods Combined incremental subgradient and proximal methods Convergence analysis Cyclic and randomized component selection |
Lecture 22 (PDF) |
Bertsekas, Dimitri. "Incremental Gradient, Subgradient, and Proximal Methods for Convex Optimization: A Survey." (PDF) Sra, Suvrit, Sebastian Nowozin, and Stephen Wright, eds. |

Review of subgradient methods Application to differentiable problems: gradient projection
Iteration complexity issues Complexity of gradient projection Projection method with extrapolation Optimal algorithms |
Lecture 23 (PDF) | |

Gradient proximal minimization method Nonquadratic proximal algorithms Entropy minimization algorithm Exponential augmented Lagrangian method Entropic descent algorithm |
Lecture 24 (PDF) |
Beck, Amir, and Marc Teboulle. "Gradient-Based Algorithms with Applications to Signal-Recovery Problems." In Beck, Amir, and Marc Teboulle. "Mirror Descent and Nonlinear Projected Subgradient Methods for Convex Optimization." Bertsekas, Dimitri. |

Convex analysis and duality Convex optimization algorithms |
Lecture 25 (PDF - 2.0MB) |