ECE 555: Control of Stochastic Systems (Spring 2019)
- Important dates
- In-class midterm: Thu Mar 14
Final exam: take-home
- May 1
- Lecture notes for week 13 (scribed by Yasutaka Narazaki and Josh Hanson) and week 14 (scribed by Maxim Raginsky) are up.
- April 23
- Homework 5 is out, due by the end of the day on Tue, Apr 30.
- April 22
- Lecture notes for week 12 are up (scribed by Liming Wang and Neng Wan).
- April 18
- There are some bugs in Homework 4 (thanks to those who spotted them), the updated version is posted now. In the meantime, the due date has been extended to Tue, Apr 23.
- There will be no lecture on Thu Apr 25 or Tue Apr 30. Instead, we will have a make-up class on Mon, Apr 28, from 3:30pm to 4:50pm (the usual class time) in CSL 154.
- April 17
- Lecture notes for week 11 are up (scribed by Jin Kim and William Wei).
- April 9
- Homework 4 is out, due by the end of the day on Thursday, April 18.
- April 2
- Lecture notes for weeek 10 are up (scribed by Maxim Raginsky).
- March 12
- Lecture notes for week 8 are up (scribed by Yasutaka Narazaki and Belinda Tzen).
- don't forget the in-class midterm on Thursday, March 14!
- March 4
- Lecture notes for week 6 are up (scribed by Yifeng Chu and Josh Hanson).
- February 26
- Homework 3 is out, due by the end of the day on Thursday, March 7.
- Lecture notes for week 5 are up (scribed by Wei Wei and Siddhartha Satpathi).
- February 18
- Lecture notes for week 4 are up (scribed by Seo Taek Kong and Harsh Gupta).
- There were a few typos in Homework 2 (thanks to Jin Kim for pointing them out). They have been corrected, and the due date has been extended to Thursday, Feb 21.
- February 12
- Lecture notes for week 3 are up (scribed by Neng Wan and Liming Wang).
- February 7
- Homework 2 is out, due by the end of the day on Tuesday, February 19.
- February 4
- Lecture notes for week 2 are up (scribed by Jin Kim and Allan Axelrod).
- January 24
- Homework 1 is out, due by the end of the day on Tuesday, February 5. All homework submissions will be done through Compass, see coursework info for instructions.
- January 22
Welcome! Watch this space for all important course-related announcements.
About this course
- What is this?
- ECE 555: Control of Stochastic Systems is a graduate-level introduction to the mathematics of stochastic control. The topics include controlled Markov processes, both in discrete and in continuous time, dynamic programming, complete and partial observations, linear and nonlinear filtering, and approximate dynamic programming.
- The core material will come from lectures. Here is a partial list of books and lecture notes I find useful:
- D.P. Bertsekas, Dynamic Programming and Optimal Control, vol. 1, Athena Scientific, 4th edition, 2017
- W.H. Fleming and R.W. Rishel, Deterministic and Stochastic Optimal Control, Springer, 1975
- P.R. Kumar and P. Varaiya, Stochastic Systems: Estimation, Identification, and Adaptive Control, Prentice Hall, 1986
- A. Mahajan, Stochastic Control and Decision Theory, course notes, McGill University
- M.L. Puterman, Markov Decision Processes: Discrete Stochastic Dynamic Programming, Wiley, 2005
- R. Weber, Optimization and Control, course notes, University of Cambridge
- Grades will be based on scribe notes (10%), regular written homeworks (40%), in-class midterm (20%), and a final exam (30%).
Regular weekly schedule
- Tue Thu 3:30–4:50pm, 2017 ECE Building
| instructor || Thu 2:00-3:00pm, 162 CSL
- Assigned on Thursdays, due by the end of the day on the due date. All homework must be typeset in LaTeX and uploaded to Compass (see coursework info).
Under normal circumstances, graded homework should be returned within 10 days of submission.