The lecture notes for this course were prepared by Alexander Rakhlin and Wen Dong, students in the class.

Complete set of lecture notes in one file (PDF - 4.8 MB)

LEC # | TOPICS |
---|---|

1 | Introduction |

2 | Voting classifiers, training error of boosting (PDF) |

3 | Support vector machines (SVM) (PDF) |

4 | Generalization error of SVM (PDF) |

5 | One dimensional concentration inequalities. Bennett’s inequality (PDF) |

6 | Bernstein’s inequality (PDF) |

7 | Hoeffding, Hoeffding-Chernoff, and Khinchine inequality (PDF) |

8 | Vapnik-Chervonenkis classes of sets (PDF) |

9 | Properties of VC classes of sets (PDF) |

10 | Symmetrization. Pessimistic VC inequality (PDF) |

11 | Optimistic VC inequality (PDF) |

12 | VC subgraph classes of functions. Packing and covering numbers (PDF) |

13 | Covering numbers of the VC subgraph classes (PDF) |

14 | Kolmogorov’s chaining method. Dudley’s entropy integral (PDF) |

15 | More symmetrization. Generalized VC inequality (PDF) |

16 | Consequences of the generalized VC inequality (PDF) |

17 | Covering numbers of the convex hull (PDF) |

18 | Uniform entropy condition of VC-hull classes (PDF) |

19 | Generalization error bound for VC-hull classes (PDF) |

20 | Bounds on the generalization error of voting classifiers (PDF) |

21 | Bounds on the generalization error of voting classifiers (cont.) (PDF) |

22 | Bounds on the generalization error of voting classifiers (cont.) (PDF) |

23 | Bounds in terms of sparsity (PDF) |

24 | Bounds in terms of sparsity (cont.) (example) (PDF) |

25 | Martingale-difference inequalities (PDF) |

26 | Comparison inequality for Rademacher processes (PDF) |

27 | Application of martingale inequalities. Generalized martingale inequalities (PDF) |

28 | Generalization bounds for neural networks (PDF) |

29 | Generalization bounds for neural networks (cont.) (PDF) |

30 | Generalization bounds for kernel methods (PDF) |

31 | Optimistic VC inequality for random classes of sets (PDF) |

32 | Applications of random VC inequality to voting algorithms and SVM (PDF) |

33 | Talagrand’s convex-hull distance inequality (PDF) |

34 | Consequences of Talagrand’s convex-hull distance inequality (PDF) |

35 | Talagrand’s concentration inequality for empirical processes (PDF) |

36 | Talagrand’s two-point inequality (PDF) |

37 | Talagrand’s concentration inequality for empirical processes (PDF) |

38 | Applications of Talagrand’s concentration inequality (PDF) |

39 | Applications of talagrand’s convex-hull distance inequality. Bin packing (PDF) |

40 | Entropy tensorization inequality. Tensorization of Laplace transform (PDF) |

41 | Application of the entropy tensorization technique (PDF) |

42 | Stein’s method for concentration inequalities (PDF) |