The lecture notes for this course were prepared by Alexander Rakhlin and Wen Dong, students in the class.
Complete set of lecture notes in one file (PDF - 4.8 MB)
LEC # | TOPICS |
---|---|
1 | Introduction |
2 | Voting classifiers, training error of boosting (PDF) |
3 | Support vector machines (SVM) (PDF) |
4 | Generalization error of SVM (PDF) |
5 | One dimensional concentration inequalities. Bennett’s inequality (PDF) |
6 | Bernstein’s inequality (PDF) |
7 | Hoeffding, Hoeffding-Chernoff, and Khinchine inequality (PDF) |
8 | Vapnik-Chervonenkis classes of sets (PDF) |
9 | Properties of VC classes of sets (PDF) |
10 | Symmetrization. Pessimistic VC inequality (PDF) |
11 | Optimistic VC inequality (PDF) |
12 | VC subgraph classes of functions. Packing and covering numbers (PDF) |
13 | Covering numbers of the VC subgraph classes (PDF) |
14 | Kolmogorov’s chaining method. Dudley’s entropy integral (PDF) |
15 | More symmetrization. Generalized VC inequality (PDF) |
16 | Consequences of the generalized VC inequality (PDF) |
17 | Covering numbers of the convex hull (PDF) |
18 | Uniform entropy condition of VC-hull classes (PDF) |
19 | Generalization error bound for VC-hull classes (PDF) |
20 | Bounds on the generalization error of voting classifiers (PDF) |
21 | Bounds on the generalization error of voting classifiers (cont.) (PDF) |
22 | Bounds on the generalization error of voting classifiers (cont.) (PDF) |
23 | Bounds in terms of sparsity (PDF) |
24 | Bounds in terms of sparsity (cont.) (example) (PDF) |
25 | Martingale-difference inequalities (PDF) |
26 | Comparison inequality for Rademacher processes (PDF) |
27 | Application of martingale inequalities. Generalized martingale inequalities (PDF) |
28 | Generalization bounds for neural networks (PDF) |
29 | Generalization bounds for neural networks (cont.) (PDF) |
30 | Generalization bounds for kernel methods (PDF) |
31 | Optimistic VC inequality for random classes of sets (PDF) |
32 | Applications of random VC inequality to voting algorithms and SVM (PDF) |
33 | Talagrand’s convex-hull distance inequality (PDF) |
34 | Consequences of Talagrand’s convex-hull distance inequality (PDF) |
35 | Talagrand’s concentration inequality for empirical processes (PDF) |
36 | Talagrand’s two-point inequality (PDF) |
37 | Talagrand’s concentration inequality for empirical processes (PDF) |
38 | Applications of Talagrand’s concentration inequality (PDF) |
39 | Applications of talagrand’s convex-hull distance inequality. Bin packing (PDF) |
40 | Entropy tensorization inequality. Tensorization of Laplace transform (PDF) |
41 | Application of the entropy tensorization technique (PDF) |
42 | Stein’s method for concentration inequalities (PDF) |