ECE 555: Control of Stochastic Systems (Spring 2019)
 Resources

 Important dates
 Inclass midterm: Thu Mar 14
Final exam: takehome
Announcements
 May 1

 Lecture notes for week 13 (scribed by Yasutaka Narazaki and Josh Hanson) and week 14 (scribed by Maxim Raginsky) are up.
 April 23

 Homework 5 is out, due by the end of the day on Tue, Apr 30.
 April 22

 Lecture notes for week 12 are up (scribed by Liming Wang and Neng Wan).
 April 18

 There are some bugs in Homework 4 (thanks to those who spotted them), the updated version is posted now. In the meantime, the due date has been extended to Tue, Apr 23.
 There will be no lecture on Thu Apr 25 or Tue Apr 30. Instead, we will have a makeup class on Mon, Apr 28, from 3:30pm to 4:50pm (the usual class time) in CSL 154.
 April 17

 Lecture notes for week 11 are up (scribed by Jin Kim and William Wei).
 April 9

 Homework 4 is out, due by the end of the day on Thursday, April 18.
 April 2

 Lecture notes for weeek 10 are up (scribed by Maxim Raginsky).
 March 12

 Lecture notes for week 8 are up (scribed by Yasutaka Narazaki and Belinda Tzen).
 don't forget the inclass midterm on Thursday, March 14!
 March 4

 Lecture notes for week 6 are up (scribed by Yifeng Chu and Josh Hanson).
 February 26

 Homework 3 is out, due by the end of the day on Thursday, March 7.
 Lecture notes for week 5 are up (scribed by Wei Wei and Siddhartha Satpathi).
 February 18

 Lecture notes for week 4 are up (scribed by Seo Taek Kong and Harsh Gupta).
 There were a few typos in Homework 2 (thanks to Jin Kim for pointing them out). They have been corrected, and the due date has been extended to Thursday, Feb 21.
 February 12

 Lecture notes for week 3 are up (scribed by Neng Wan and Liming Wang).
 February 7

 Homework 2 is out, due by the end of the day on Tuesday, February 19.
 February 4

 Lecture notes for week 2 are up (scribed by Jin Kim and Allan Axelrod).
 January 24

 Homework 1 is out, due by the end of the day on Tuesday, February 5. All homework submissions will be done through Compass, see coursework info for instructions.
 January 22


Welcome! Watch this space for all important courserelated announcements.
About this course
 What is this?
 ECE 555: Control of Stochastic Systems is a graduatelevel introduction to the mathematics of stochastic control. The topics include controlled Markov processes, both in discrete and in continuous time, dynamic programming, complete and partial observations, linear and nonlinear filtering, and approximate dynamic programming.
 Materials
 The core material will come from lectures. Here is a partial list of books and lecture notes I find useful:
 D.P. Bertsekas, Dynamic Programming and Optimal Control, vol. 1, Athena Scientific, 4th edition, 2017
 W.H. Fleming and R.W. Rishel, Deterministic and Stochastic Optimal Control, Springer, 1975
 P.R. Kumar and P. Varaiya, Stochastic Systems: Estimation, Identification, and Adaptive Control, Prentice Hall, 1986
 A. Mahajan, Stochastic Control and Decision Theory, course notes, McGill University
 M.L. Puterman, Markov Decision Processes: Discrete Stochastic Dynamic Programming, Wiley, 2005
 R. Weber, Optimization and Control, course notes, University of Cambridge
 Coursework
 Grades will be based on scribe notes (10%), regular written homeworks (40%), inclass midterm (20%), and a final exam (30%).
Regular weekly schedule
 Lectures
 Tue Thu 3:30–4:50pm, 2017 ECE Building

Office hours:

instructor  Thu 2:003:00pm, 162 CSL

 Homework
 Assigned on Thursdays, due by the end of the day on the due date. All homework must be typeset in LaTeX and uploaded to Compass (see coursework info).
Under normal circumstances, graded homework should be returned within 10 days of submission.