ECE 555: Lecture Schedule

The schedule will be updated and revised as the course progresses. Links to scribe lectures are on the left.

Markov Decision Processes in discrete time

Tue Jan 15
Introduction and administrivia
Control as optimization over time
Stochastic optimization, static case:
Thu Jan 17
[week 1 notes]
Controlled Markov processes
Policies and their expected cost
Optimality of Markov policies
Tue Jan 22
Dynamic programming Example: equipment replacement problem
Thu Jan 24
[week 2 notes]
Monotonicity of value functions
Tue Jan 29
Markov decision processes with general state and action spaces
Thu Jan 31
[week 3 notes]
The finite-horizon Linear Quadratic Regulator (LQR) problem
Tue Feb 5
MDPs with partial observations: the finite case
Thu Feb 7
[week 4 notes]
MDPs with partial observations, continued
Tue Feb 12
MDPs with partial observations, continued
Thu Feb 14
[week 5 notes]
The two-armed bandit problem
Tue Feb 19
Stochastic control over an infinite horizon
Thu Feb 21
[week 6 notes]
Discounted cost, continued
Tue Feb 26
Average-cost optimality criterion
Thu Feb 28
Tue Mar 5
Linear Quadratic Regulator under the average-cost criterion
Thu Mar 7
[week 8 notes]
Linear-Quadratic-Gaussian (LQG) problem with partial observations
Tue Mar 12

Thu Mar 14
In-Class Midterm

Tue Mar 19
Thu Mar 21

Tue Mar 26
Thu Mar 28
[week 10 notes]
Partially observed LQG

Markov Decision Processes in continuous time

Tue Apr 2
Thu Apr 4
[week 11 notes]
Continuous-time Markov processes
Tue Apr 9
Thu Apr 11
[week 12 notes]
Ito calculus and controlled diffusion processes
Tue Apr 16
Thu Apr 18
[week 13 notes]
Markov decision processes in continuous time: examples
Tue Apr 23
Mon Apr 29
[week 14 notes]
Nonlinear filtering in continuous time