**(Undergrad Desc)**

Math 632 gives an introduction to Markov chains and Markov processes with discrete state spaces and their applications. Particular models studied include birth-death chains, queuing models, random walks and branching processes. Selected topics from renewal theory, martingales, and Brownian motion are also included, but vary from semester to semester to meet the needs of different audiences. Those looking for a similar course, though one with more of a focus on biological applications and computational methods, should look at Math 605.

Math 605

Math 635, 733, 734, 735

- Markov Chains
- transition functions and related computations
- classification of states: recurrence, transcience, irreducibility, periodicity
- examples: queuing, birth-death chains, branching, random walks

- Limiting Behavior of Markov Chains
- the main limit theorem and stationary distributions
- absorption probabilities
- further recurrence criteria

- Continuous Time Markov Chains
- definitions and examples (Poisson process)
- structure of a Markov process: waiting times and jumps
- the Kolmogorov differential equations
- limit theory
- birth-death processes and other examples

- Selected Topics
- renewal theory and applications
- martingales
- a first look at Brownian motion and some applications

** **

** **

**(Roch F 2014) Prerequisites: **Good knowledge of undergraduate probability at the level of UW-Madison Math 431 (or an equivalent course: Math 331, Stat 309, 311, 313) is required. This means familiarity with basic probability models, random variables and their probability mass functions and distributions, expectations, joint distributions, independence, conditional probabilities, the law of large numbers and the central limit theorem. If you need a thorough review of basics, the textbook Introduction to Probability by Anderson, Seppalainen, and Valko is recommended. Some 500-level work in mathematics is recommended for background, preferably in analysis (521).**Course content: **Math 632 gives an introduction to Markov chains and Markov processes with discrete state spaces and their applications. Particular models studied include birth-death chains, queuing models, random walks, branching processes and renewal processes. If time permits, we will also discuss Brownian motion.**Note**: A typical advanced math course follows a strict theorem-proof format. 632 is not of this type. Mathematical theory is discussed in a precise fashion but only some results can be rigorously proved in class. This is a consequence of time limitations and the desire to leave measure theory outside the scope of this course. Interested students can find the proofs in the textbook. For a thoroughly rigorous probability course students should sign up for the graduate probability sequence 831-832.

**(Shinault/Wood, 2014)**

- Markov Chains
- transition functions and related computations
- classification of states: recurrence, transcience, irreducibility, periodicity
- examples: queuing, birth-death chains, branching, random walks

- Limiting Behavior of Markov Chains
- the main limit theorem and stationary distributions
- absorption probabilities
- further recurrence criteria

- Continuous Time Markov Chains
- definitions and examples (Poisson process)
- structure of a Markov process: waiting times and jumps
- the Kolmogorov differential equations
- limit theory
- birth-death processes and other examples

- Selected Topics
- renewal theory and applications
- a first look at Martingales