ECE 563: Lecture Schedule

The schedule will be updated and revised as the course progresses. Required reading will be indicated on the left.

Information Measures

Tue Aug 29
Introduction and administrivia
Entropy and divergence
Thu Aug 31
Mutual information
Tue Sep 5
Entropy and combinatorics:
Thu Sep 7
Sufficient statistics
Variational representations of information measures
Tue Sep 12
Variational representations wrap-up
Extremization of mutual information: capacity, minimax characterization
Thu Sep 14
Extremization of mutual information: information radius, Gaussian saddle-point
Tue Sep 19
Single-letterization, probability of error
Converse bounds via Fano's inequality

Lossless Data Compression

Thu Sep 21
Variable-length lossless compression
Tue Sep 26
Fixed-length (almost lossless) compression
Slepian-Wolf
Thu Sep 28
Compressing stationary ergodic sources
Tue Oct 3
Universal compression
Thu Oct 5
No lecture: Allerton conference
Mon Oct 9
Midterm I, 7:00-9:30pm

Binary Hypothesis Testing

Tue Oct 10
Hypothesis testing, Neyman-Pearson formulation
Thu Oct 12
Asymptotic regimes: Stein and Chernoff
Tue Oct 17
Basics of large deviations
Thu Oct 19
Large deviations: exponents, information projection, Sanov's theorem
Tue Oct 24
Large deviations wrap-up
Thu Oct 26
More asymptotics

Channel Coding

Tue Oct 31
Happy Halloween! 👻
Basic results on channel coding
Operational criteria, weak converse
Thu Nov 2
Channel coding: achievability bounds
Tue Nov 7
Linear codes
Channel capacity
Thu Nov 9
Channels with input constraints
Gaussian channels
Mon Nov 13
Midterm II, 7:00-9:30pm
Tue Nov 14
Energy per bit
Continuous-time channels
Thu Nov 16
Source-channel separation
Tue Nov 21
Thanksgiving break
Thu Nov 23
Thanksgiving break

Lossy Data Compression

Tue Nov 28
Rate-distortion theory
Scalar and vector quantization
Thu Nov 30
Rate-distortion theory: achievability bounds
Tue Dec 5
Evaluating R(D)
Thu Dec 7
Lossy source-channel separation
Tue Dec 12
Review