ECE 498MR: Introduction to Stochastic Systems (Spring 2016)

About  |  Schedule  |  Coursework


The schedule will be updated and revised as the course progresses. To get a rough idea of what to expect, consult the course syllabus. Required reading in the course notes is indicated on the left; key topics will be highlighted.
Wed, Jan 20

Introduction and administrivia.

  • What are stochastic signals and systems?
  • Noise, uncertainty, randomness
  • Descriptions of stochastic systems: declarative vs. imperative
Mon, Jan 25
Wed, Jan 27
Mon, Feb 1

First look: Markov chains as stochastic systems

  • Systems with state: deterministic and stochastic
  • Markov chains as noise-driven systems with state
  • Motivating examples: simple random walk on the integers, two-state Markov chain.
  • Descriptions of Markov chains: imperative and declarative
  • Analysis in the space of probabilities via z-transforms (probability-generating functions) and matrix multiplication
  • Equilibrium distributions and the PageRank algorithm
Wed, Feb 3
Mon, Feb 8
Wed, Feb 10
Mon, Feb 15

Random signals and probabilistic systems

  • Random processes as signals (in discrete and continuous time)
  • Random walks: independent increments, Markov property
  • Wiener and Poisson processes as continuous-time limits of random walks
  • Stationarity: strong and weak
Wed, Feb 17
Mon, Feb 22
Wed, Feb 24
Mon, Feb 29
Mon, Mar 7
Wed, Mar 9
Mon, Mar 14

Following the dynamics

  • Moments, auto- and cross-correlation in time and frequency domain; spectral methods; white and colored noise
  • Input-output relations; theorems of Bussgang and Campbell; fluctuation-dissipation relations
  • Basic analysis of convergence and stability via Lyapunov (or potential) functions
  • Case studies: average consensus and PageRank revisited

Wed, Mar 16
Mon, Mar 28
Wed, Mar 30


  • Dynamical view: evolution of uncertainty and information in time.
  • Bayesian filtering and Hidden Markov Models.
  • A glimpse of Poisson systems and queues.
Mon, Apr 4
In-Class Midterm
Wed, Apr 6
Mon, Apr 11
Wed, Apr 13
Mon, Apr 18
  • Noise mechanisms in physical systems: shot noise, Johnson-Nyquist noise, van der Ziel (1/f) noise, amplifier noise.
  • Case studies: discovery of cosmic microwave background radiation by Penzias and Wilson; noise and Bayesian filtering in remote sensing systems.

Mon, Apr 18
Wed, Apr 20
Mon, Apr 25
Randomness and determinism
  • Law of Large Numbers and the Central Limit Theorem through the lens of linear systems.
  • Variance reduction by averaging (examples: invention of least squares; diversification in financial portfolios following Harry Markowitz; Monte Carlo simulation).
  • Large-deviation bounds via the Chernoff technique and Taylor series (example: probabilistic interpretation of multiplexing gain in telephony).
Wed, Apr 27
Mon, May 2
Wed, May 4
Feedback and control
  • Controlled Markov chains: imperative and declarative descriptions
  • Finite-horizon optimal control problem
  • Blackwell's principle of irrelevant information and dynamic programming