Meetings: TuTh 9:30AM - 10:45AM, VAN VLECK B115
Instructor: Benedek Valkó
Office: 409 Van Vleck
Email: valko at math dot wisc dot edu
Office hours: Tu 11-12 or by appointment
This is the course homepage that also serves as the syllabus for the course. Here you will find homework assignments, our weekly schedule, and updates on scheduling matters. The Mathematics Department also has a general information page on this course.
I will use the class email list to send out corrections, announcements, please check your wisc.edu email from time to time.
Math 632 is a course on basic stochastic processes and
applications with an emphasis on problem solving. Topics will
include discrete-time Markov chains, Poisson point
processes, continuous-time Markov chains, and renewal processes.
The material will be treated in a mathematically precise fashion
although some proofs will be skipped due to time limitations.
To go beyond Math 632 in probability you should consider taking
Math 635 - Introduction to Brownian Motion and Stochastic
Good knowledge of undergraduate probability at the level of
UW-Madison Math 431 (or an equivalent course: Math 331, Stat 309,
311, 313) is required. This means familiarity with basic
probability models, random variables and their probability mass
functions and distributions, expectations, joint distributions,
independence, conditional probability and expectation, the law of
large numbers and the central limit theorem.
If you need a thorough review of basics, the textbook A First
Course in Probability by S. Ross is recommended. I will also
upload a set of lecture notes on probability at Learn@UW which is
currently being developed for Math 431. For those, who just need a
quick refresher, this write-up
summarizes some of the basic concepts of probability theory you
should be familiar with. I
strongly suggest that you review this before the first class.
The appendix of the textbook also contains an overview of the
Familiarity with calculus and linear algebra is also needed. Some 500-level work in mathematics is recommended for background, preferably in analysis (521).
Course grades will be based on home work assignments (20%), two
midterm exams (20% each) and the final exam (40%).
You will be allowed to use a single written page of notes on the midterm exams and two written pages on the final exam. Calculators and other electronic devices will not be permitted during the exams.
The midterm exams will be evening exams scheduled on Wednesdays.
Homework assignments will be posted on the Learn@UW site of the
course. Weekly homework assignments are due Thursdays at the
beginning of the class.
Note that the first homework assignment is due on September 4
(the first Thursday of the semester!). In order to solve the
problems on the first assignment you will only need the concepts
learned in Math 431 (or any other introductory probability
course). If you are having a hard time with this assignment then
you should definitely spend some time reviewing the this
||Suggested reading for next
|Review of basic concepts
of probability. Discrete Markov chains: definitions and
the transition probability matrix, multistep probabilities,
posted notes, Sections 1.1-1.2
|Classification of states,
strong Markov property, transience and recurrence, closed
and irreducible sets
|Stationary distributions, periodicity
Sections 1.4, 1.5
|Limit behavior, expected return time,
examples, birth and death process, proofs of the limit
Sections 1.5-1.7, 1.10
|Exit times and exit distributions, the
Metropolis-Hastings algorithm, extinction probability in a
Sections 1.8, 1.9, 1.10
|Properties of the exponential distribution,
the Poisson process, number of arrivals in an interval, the
spatial Poisson process
Sections 2.1, 2.2
|Compound Poisson process, transformations of
First midterm exam
|Finish up Poisson process. Renewal processes.
Sections 2.4, 3.1
|Age and residual life
|Continuous time Markov chains, Markov
property in continuous time, the embedded discrete time MC,
|Examples, the Poisson clock construction,
Kolmogorov's backward and forward equations
the Kolmogorov equations, stationary distribution,
Second midterm exam
|Stationary distribution, detailed balance
|Birth and death processes, exit times,
|Brief introduction to Brownian motion, review