ECE 299: Statistical Learning Theory (Spring 2011)

About  |  Schedule  |  References  |  Coursework


The schedule will be updated and revised as the course progresses. Each topic will come with links to reference materials; key references will be highlighted.
Wed, Jan 19

Introduction, history, overview, and administrivia.

Mon, Jan 24
Wed, Jan 26
Mon, Jan 31

Concentration inequalities: Markov, Chebyshev, McDiarmid (bounded differences inequality), examples

Wed, Feb 02
Mon, Feb 07
Wed, Feb 09
Mon, Feb 14

Formulaton of the learning problem: concept and function learning; agnostic (model-free) learning; consistency; Probably Approximately Correct (PAC) learning; Empirical Risk Minimization

Wed, Feb 16
Mon, Feb 21

Empirical Risk Minimization: abstract risk bounds and Rademacher averages -- stochastic inequalities for ERM; Rademacher averages (structural results, Finite Class Lemma); introduction to VC classes

Wed, Feb 23
Vapnik-Chervonenkis classes: shatter coefficients; VC dimension; examples of VC classes; Sauer-Shelah lemma; implication for Rademacher averages
Mon, Feb 28
Wed, Mar 02
Case study: empirical quantizer design
Mar 14
Mar 16
Mar 21
Mar 23
Binary classification: bounds for simple VC classes (linear and generalized linear discriminant rules); margin-based bounds; reproducing kernel Hilbert spaces and kernel machines; convex risk minimization
Mar 28
Mar 30
Regression with quadratic loss

Mon, Apr 04
Mon, Apr 11
Wed, Apr 13
Case study: stochastic simulation via Rademacher bootstrap