Math 833 - Continuous Time Markov Processes

Spring 2014

Meetings: TR 11:00-12:15, Van Vleck B131
Instructor: Benedek Valkó
Office: 409 Van Vleck
Phone: 263-2782
Email: valko at math dot wisc dot edu
Office hours: by appointment

I will use the class email list to send out corrections, announcements, please check your email from time to time.

Course description

The course will focus on an important class of stochastic processes: continuous time Markov processes. We will discuss specific examples (e.g. Brownian motion, Markov processes with countable state space) and the general theory (semigroup theory, infinitesimal generators, Hille-Yosida theory). The second part of the course will provide an introduction to the theory of interacting particle systems.


 733 (old 831) or an equivalent probability course (with measure theory). It would be useful to have some background from discrete Markov chains (632 or 734), but we can review this within the course.


Liggett: Continuous Time Markov Processes - An Introduction

A couple of other useful references:

We might also use additional resources.

Course content

We will cover the Chapters 2-4 from Liggett:

In the second half of the semester we will look at various other topics related to interacting particle systems: hydrodinamic scaling limits, central limit theorems,...

If you have a another topic you would like to learn about: let me know!


The final grade will be based on homework assignments. There is no final exam in the course.

Homework assignments

First assignment: Problems 2.32, 2.36,  2.55, 2.65 from Liggett. Due date: February 27
Second assignment: hw2 Due date: March 27
Third assignment:
Problems 4.24, 4.36 a) (use the reflection principle), 4.46 from Liggett. Due date: May 6

Weekly schedule

1. week (1/21, 1/23): Basic definitions, examples, Markov processes on discrete state space
2. week (1/30): Differentiability of the transition function
3. week (2/4, 2/6): Blackwell's example, the minimal solution of the backward equation with a given Q-matrix
4. week (2/11, 2/13): Constructing the distribution of the MC from the Q-matrix, Feller processes, the resolvent
5. week (2/18, 2/20): The generator, the probability semigroup and generator of a Feller process.  From Feller process to infinitesimal description.
6. week (2/25, 2/27): From infinitesimal description to Feller process. Closure of an operator.
7. week (3/6): guest lecture by D. Anderson
8. week (3/11, 3/13): martingales and Feller processes, stationary distributions for Feller processes, duality
9. week (3/25, 3/27): superposition of processes, the Feynman-Kac formula, interacting particle systems introduction
10. week (4/1, 4/3): existence theorem for spin systems, ergodicity of spin systems, coupling
11. week (4/8, 4/10): duality in the voter model, stationary measures for the voter model
12. week (4/15, 4/17): a closer look at the contact process, the exclusion process
13. week (4/22, 4/24): stationary distributions of the exclusion process, symmetric and translation invariant cases
14. week (4/29, 5/1): a  first look at hydrodynamic limits: the case of independent random walkers