First look: Markov chains as stochastic systems
- Systems with state: deterministic and stochastic
- Markov chains as noise-driven systems with state
- Motivating examples: simple random walk on the integers, two-state Markov chain.
- Descriptions of Markov chains: imperative and declarative
- Analysis in the space of probabilities via z-transforms (probability-generating functions) and matrix multiplication
- Equilibrium distributions and the PageRank algorithm