ECE 555: Control of Stochastic Systems (Fall 2025)

Announcements

November 18

• Homework 6 has been released, due by the end of the day on Thursday, December 4.

November 13

• Final project info has been released, check the coursework page. The final projects will be due by the end of the day on December 18.

November 6

• Homework 5 has been released, due by the end of the day on Thursday, November 13.

October 30

• Homework 4 has been released, due by the end of the day on Thursday, November 6.

October 16

• The midterm exam will be held in class on Tuesday, October 21. You can use your notes. The exam will cover the material before this week's lectures.
• There will be no class on Thursday, October 23.

October 9

• Homework 3 has been released, due by the end of the day on Thursday, October 16.

September 25

• Homework 2 has been released, due by the end of the day on Thursday, October 2.

September 23

• The midterm exam will be held in class on Tuesday, October 21.

September 12

• Because of the Allerton Conference, there will be no lecture on Tuesday, September 16 and no TA or instructor office hours that week. There will be regular lecture on Thursday, September 18.

September 4

• Homework 1 has been released, due by the end of the day on Thursday, September 11.

August 22

• Welcome! Watch this space for all important course-related announcements.

About this course

What is this?

ECE 555: Control of Stochastic Systems is a graduate-level introduction to the mathematics of stochastic control. The topics include controlled Markov processes, both in discrete and in continuous time, dynamic programming, complete and partial observations, linear and nonlinear filtering, and approximate dynamic programming.

Materials

The core material will come from lectures. Here is a partial list of books and lecture notes I find useful:

• D.P. Bertsekas, Dynamic Programming and Optimal Control, vol. 1, Athena Scientific, 4th edition, 2017
• W.H. Fleming and R.W. Rishel, Deterministic and Stochastic Optimal Control, Springer, 1975
• P.R. Kumar and P. Varaiya, Stochastic Systems: Estimation, Identification, and Adaptive Control, Prentice Hall, 1986
• A. Mahajan, Stochastic Control and Decision Theory, course notes, McGill University
• M.L. Puterman, Markov Decision Processes: Discrete Stochastic Dynamic Programming, Wiley, 2005
• R. Weber, Optimization and Control, course notes, University of Cambridge

Coursework

Grades will be based regular written homework sets (40%), in-class midterm (20%), and a final project (40%).

Regular weekly schedule

Lectures

Tue Thu 11am–12:20pm, 3020 ECE Building

Office hours:

instructor	Thu 9:30-10:30am, 162 CSL
TA	Wed 3:00-4:00pm, 2036 ECEB

Homework

Assigned on Thursdays, due by the end of the day on the due date. All homework must be typeset in LaTeX and uploaded to Gradescope (see coursework info).
Under normal circumstances, graded homework should be returned within 10 days of submission.