Department of Mathematics

Van Vleck Hall, 480 Lincoln Drive, Madison, WI

Math 632: Intro to Stochastic Processes

(Undergrad Desc)

Background and Goals: 

Math 632 gives an introduction to Markov chains and Markov processes with discrete state spaces and their applications. Particular models studied include birth-death chains, queuing models, random walks and branching processes. Selected topics from renewal theory, martingales, and Brownian motion are also included, but vary from semester to semester to meet the needs of different audiences.   Those looking for a similar course, though one with more of a focus on biological applications and computational methods, should look at Math 605.

Alternatives: 

Math 605

Subsequent Courses: 

Math 635, 733, 734, 735

Course Content: 
  • Markov Chains
    • transition functions and related computations
    • classification of states: recurrence, transcience, irreducibility, periodicity
    • examples: queuing, birth-death chains, branching, random walks
  • Limiting Behavior of Markov Chains
    • the main limit theorem and stationary distributions
    • absorption probabilities
    • further recurrence criteria
  • Continuous Time Markov Chains
    • definitions and examples (Poisson process)
    • structure of a Markov process: waiting times and jumps
    • the Kolmogorov differential equations
    • limit theory
    • birth-death processes and other examples
  • Selected Topics
    • renewal theory and applications
    • martingales
    • a first look at Brownian motion and some applications

 

 

(Roch F 2014) Prerequisites: Good knowledge of undergraduate probability at the level of UW-Madison Math 431 (or an equivalent course: Math 331, Stat 309, 311, 313) is required. This means familiarity with basic probability models, random variables and their probability mass functions and distributions, expectations, joint distributions, independence, conditional probabilities, the law of large numbers and the central limit theorem. If you need a thorough review of basics, the textbook A First Course in Probability by S. Ross is recommended.Some 500-level work in mathematics is recommended for background, preferably in analysis (521).Course content: Math 632 gives an introduction to Markov chains and Markov processes with discrete state spaces and their applications. Particular models studied include birth-death chains, queuing models, random walks, branching processes and renewal processes. If time permits, we will also discuss Brownian motion.Note: A typical advanced math course follows a strict theorem-proof format. 632 is not of this type. Mathematical theory is discussed in a precise fashion but only some results can be rigorously proved in class. This is a consequence of time limitations and the desire to leave measure theory outside the scope of this course. Interested students can find the proofs in the textbook. For a thoroughly rigorous probability course students should sign up for the graduate probability sequence 831-832.   

(Shinault/Wood, 2014)

  • Markov Chains
    • transition functions and related computations
    • classification of states: recurrence, transcience, irreducibility, periodicity
    • examples: queuing, birth-death chains, branching, random walks
  • Limiting Behavior of Markov Chains
    • the main limit theorem and stationary distributions
    • absorption probabilities
    • further recurrence criteria
  • Continuous Time Markov Chains
    • definitions and examples (Poisson process)
    • structure of a Markov process: waiting times and jumps
    • the Kolmogorov differential equations
    • limit theory
    • birth-death processes and other examples
  • Selected Topics
    • renewal theory and applications
    • a first look at Martingales
credits: 
3. (N-A)
semester: 
FallSpring
prereqs: 
Math 431, or Stat 309 and 310, or Stat 311 and 312, or Stat 313 or 314. Also, a high level of "mathematical maturity" is very useful. Such "maturity" is often gained in a 500 level course, with 521 being a good choice.

UW-Madison Department of Mathematics
Van Vleck Hall
480 Lincoln Drive
Madison, Wi  53706

(608) 263-3054

Contact Us

Got a question about
accessibility, content
or structure of this website?
webmaster@math.wisc.edu