Fall 2014
Meetings: TuTh 9:30AM  10:45AM, VAN VLECK B115
Instructor: Benedek Valkó
Office: 409 Van Vleck
Phone: 2632782
Email: valko at math dot wisc dot edu
Office hours: Tu 1112 or by appointment
This is the course homepage that also serves as the syllabus for the course. Here you will find homework assignments, our weekly schedule, and updates on scheduling matters. The Mathematics Department also has a general information page on this course.
I will use the class email list to send out corrections, announcements, please check your wisc.edu email from time to time.
Math 632 is a course on basic stochastic processes and
applications with an emphasis on problem solving. Topics will
include discretetime Markov chains, Poisson point
processes, continuoustime Markov chains, and renewal processes.
The material will be treated in a mathematically precise fashion
although some proofs will be skipped due to time limitations.
To go beyond Math 632 in probability you should consider taking
Math 635  Introduction to Brownian Motion and Stochastic
Calculus.
Good knowledge of undergraduate probability at the level of
UWMadison Math 431 (or an equivalent course: Math 331, Stat 309,
311, 313) is required. This means familiarity with basic
probability models, random variables and their probability mass
functions and distributions, expectations, joint distributions,
independence, conditional probability and expectation, the law of
large numbers and the central limit theorem.
If you need a thorough review of basics, the textbook A First
Course in Probability by S. Ross is recommended. I will also
upload a set of lecture notes on probability at Learn@UW which is
currently being developed for Math 431. For those, who just need a
quick refresher, this writeup
summarizes some of the basic concepts of probability theory you
should be familiar with. I
strongly suggest that you review this before the first class.
The appendix of the textbook also contains an overview of the
required theory.
Familiarity with calculus and linear algebra is also needed. Some
500level work in mathematics is recommended for background,
preferably in analysis (521).
Course grades will be based on home work assignments (20%), two
midterm exams (20% each) and the final exam (40%).
You will be allowed to use a single written page of notes on the
midterm exams and two written pages on the final exam. Calculators
and other electronic devices will not be permitted during the
exams.
The midterm exams will be evening exams scheduled on Wednesdays.
Exam dates:
Homework assignments will be posted on the Learn@UW site of the
course. Weekly homework assignments are due Thursdays at the
beginning of the class.
Note that the first homework assignment is due on September 4
(the first Thursday of the semester!). In order to solve the
problems on the first assignment you will only need the concepts
learned in Math 431 (or any other introductory probability
course). If you are having a hard time with this assignment then
you should definitely spend some time reviewing the this
writeup.
Covered
topics 
Suggested reading for next
week 

Week 1 (9/2, 9/4) 
Review of basic concepts
of probability. Discrete Markov chains: definitions and
examples, the transition probability matrix, multistep probabilities, posted notes, Sections 1.11.2 
Sections 1.31.5 
Week 2 (9/9, 9/11) 
Classification of states,
strong Markov property, transience and recurrence, closed
and irreducible sets Section 1.3 
Sections 1.41.6 
Week 3 (9/16, 9/18) 
Stationary distributions, periodicity Sections 1.4, 1.5 
Sections 1.61.7 
Week 4. (9/23, 9/25) 
Limit behavior, expected return time,
examples, birth and death process, proofs of the limit
theorems Sections 1.51.7, 1.10 
Sections 1.81.10 
Week 5. (9/30, 10/2) 
Exit times and exit distributions, the
MetropolisHastings algorithm, extinction probability in a
branching process Sections 1.8, 1.9, 1.10 
Sections 2.12.2 
Week 6 (10/7, 10/9) 
Properties of the exponential distribution,
the Poisson process, number of arrivals in an interval, the
spatial Poisson process Sections 2.1, 2.2 
Section 2.32.4 
Week 7 (10/14, 10/16) 
Compound Poisson process, transformations of
Poisson processes First midterm exam Sections 2.32.4 
Section 3.1 
Week 8 (10/21, 10/23) 
Finish up Poisson process. Renewal processes.
Sections 2.4, 3.1 
Section 3.3 
Week 9 (10/28, 10/30) 
Age and residual life Section 3.3 
Section 4.1 
Week 10 (11/4, 11/6) 
Continuous time Markov chains, Markov
property in continuous time, the embedded discrete time MC,
infinitesimal rates Section 4.1 
Section 4.2 
Week 11 (11/11, 11/13) 
Examples, the Poisson clock construction,
Kolmogorov's backward and forward equations Section 4.14.2 
Section 4.3 
Week 12 (11/18, 11/20) 
Solving
the Kolmogorov equations, stationary distribution,
limiting behavior Second midterm exam Section 4.3 
Section 4.3 
Week 13 (11/25) 
Stationary distribution, detailed balance Section 4.3 
Section 4.44.5 
Week 14 (12/2, 12/4) 
Birth and death processes, exit times,
queuing examples 

Week 15 (12/9, 12/11) 
Brief introduction to Brownian motion, review 