18.465 | Spring 2007 | Graduate

Topics in Statistics: Statistical Learning Theory

Calendar

LEC # TOPICS
1 Introduction
2 Voting classifiers, training error of boosting
3 Support vector machines (SVM)
4 Generalization error of SVM
5 One dimensional concentration inequalities. Bennett’s inequality
6 Bernstein’s inequality
7 Hoeffding, Hoeffding-Chernoff, and Khinchine inequality
8 Vapnik-Chervonenkis classes of sets
9 Properties of VC classes of sets
10 Symmetrization. Pessimistic VC inequality
11 Optimistic VC inequality
12 VC subgraph classes of functions. Packing and covering numbers
13 Covering numbers of the VC subgraph classes
14 Kolmogorov’s chaining method. Dudley’s entropy integral
15 More symmetrization. Generalized VC inequality
16 Consequences of the generalized VC inequality
17 Covering numbers of the convex hull
18 Uniform entropy condition of VC-hull classes
19 Generalization error bound for VC-hull classes
20 Bounds on the generalization error of voting classifiers
21 Bounds on the generalization error of voting classifiers (cont.)
22 Bounds on the generalization error of voting classifiers (cont.)
23 Bounds in terms of sparsity
24 Bounds in terms of sparsity (cont.) (example)
25 Martingale-difference inequalities
26 Comparison inequality for Rademacher processes
27 Application of martingale inequalities. Generalized martingale inequalities
28 Generalization bounds for neural networks
29 Generalization bounds for neural networks (cont.)
30 Generalization bounds for kernel methods
31 Optimistic VC inequality for random classes of sets
32 Applications of random VC inequality to voting algorithms and SVM
33 Talagrand’s convex-hull distance inequality
34 Consequences of Talagrand’s convex-hull distance inequality
35 Talagrand’s concentration inequality for empirical processes
36 Talagrand’s two-point inequality
37 Talagrand’s concentration inequality for empirical processes
38 Applications of Talagrand’s concentration inequality
39 Applications of talagrand’s convex-hull distance inequality. Bin packing
40 Entropy tensorization inequality. Tensorization of Laplace transform
41 Application of the entropy tensorization technique
42 Stein’s method for concentration inequalities

Course Info

Departments
As Taught In
Spring 2007
Level