Course Summary: This course is, first and foremost, an introduction to stochastic processes (models that incorporate randomness) that is equivalent in level, and to a large extent content, to Math 632. Thus, we will cover the following mathematical objects:

- discrete time Markov chains
- branching processes
- the basics of point processes,
- continuous time Markov chains,
- diffusion processes (those incorporating "Brownian motion").

You may download the Syllabus.

Time: Tuesday and Thursday, 1:00 PM - 2:15 PM.

Instructor:

Office: 617 Van
Vleck

E-mail: anderson at math dot wisc dot edu

E-mail: anderson at math dot wisc dot edu

Office hours: Tuesday,
3:30-4:30pm, Friday 1:00-2:00pm, and by appointment.

Textbook: There is no text
for the course. Instead, I will provide lecture notes,
which can be found directly below. Some optional texts,
which I am going to put on reserve in the math library, are:

- Stochastic modelling for systems biology, by Darren J. Wilkinson.
- Introduction to Stochastic processes, 2nd ed., by Gregory F. Lawler.
- Adventures in Stochastic Processes, by Sidney Resnick.

Lecture notes. I will post the lecture notes as they become available. Below are current as of April 9th, 2013.

A note to visitors to this site (i.e. not my students): Please feel free to peruse these notes. However, I ask that you please notify me of any typos, or potential improvements, as these are a work in progress and such comments would be extremely useful.

**Intended audience: **Advanced
undergraduate
students and/or graduate students in mathematics, physics,
computer science, engineering, and related disciplines with an
interest in biology, and students in biochemistry, biology, and
related disciplines, with an interest in quantitative approaches
in biology.

**
Prerequisites:** It is important that the student has taken
Calculus and an introductory probability course (at the level of
Math 331/431 or Stat 309/311). A basic knowledge of
ordinary differential equations and linear algebra is also
important. Having some experience with writing codes would
also be useful, though not strictly necessary as this can be
learned during the course.

Matlab: Each homework assignment will involve some use of MATLAB, which is a mathematical software package. If you do not already have MATLAB (and I'm guessing most, if not all, of you do not have MATLAB on your personal computers) then you can use MATLAB on any Windows machine in a University of Wisconsin computer lab. A list of the labs, with a map, is here.

Grading: In determining your final grade your work will be weighted in the following manner:

Midterm Exam
1 |
25 % |

Midterm Exam
2 |
25 % |

Final Exam | 35 % |

Homework |
15 % |

Exams

- Exam 1 time is Tuesday evening, October 15th
from 7:15 - 9:15pm. The exam will cover material up
through and including Section 3.2. The exam room is
**6203 Social Sciences**.

- Exam 2 date is scheduled for Monday evening,
November 25th from 7:15 - 9:15pm. The exam room is
**B239 in Van Vleck**.

- Final Exam is set for Monday, December 16th,
from 12:25 - 2:25PM. This can not be changed, and you
*must* take the exam at this time. The exam room is
**Ingraham 120**.

The exams may be evening exams.

- Here is a baic MATLAB file that is useful for HW1. It takes nothing as an input, computes the product of two matrices (given in the code), and returns the product matrix. Make sure you play with your code. Change up some of the numbers, perform the calculation in the command line, etc.
- Here is a MATLAB code that is helpful for Problem 3 of Appendix B.
- Here is a helpful code for Problem 4 of Appendix B that approximates the expected value of a random variable with probabilities: P{X = -1} = 1/5, P{X = 5} = 1/7, P{X = 10} = 1/2, and P{X = 100} = 11/70. Note that these values are different than what you are asked to work with.
- Here is a helpful MATLAB code for number 7 of Chapter 2.
- Here is a helpful MATLAB code for number 13 of Chapter 2. Note that most of my codes are simply comments intended to help you out. The actual code is typically very short.
- Here
is nrm_dimerSimple.m, used for lecture on 12/12. Here is
nrm_dimerPlot.m, used for lecture on 12/12.

Homework assignments: All homework is due at the beginning of class on the due date.

- Due Tuesday, Sept. 10th. Appendix A, #'s
1 - 8. Appendix B, #'s 1 - 4.

- Due Tuesday, Sept. 17th. Chapter 2, #'s
1 - 7.

- Due Tuesday, Oct. 1st. Chapter 2, #'s 8 - 13.
- Due Tuesday, Oct. 8th. Chapter 2, #'s 14 - 16.
- Due Thursday, Oct. 17th. Chapter 3, #'s 1 - 7. I am not providing a MATLAB code for number 7. You should be able to tweak a code from a previous problem. Also, note that the death rate is different in the cases X_n = 0 vs X_n > 0! Turn in your code.
- Due Tuesday, Oct. 29th. Chapter 3, #'s 8
- 13. I am not providing MATLAB code for the two
computer oriented problems. At this point, you should be
able to write these types of scripts. If you need help,
SEE ME.
**H****ere is my M****ATLAB code for problem number 1****3.**

- Due Thursday, November 14th. Chapter 4, #'s 1 - 10.
- Due Tuesday, December 10th. Chapter 5,
#'s 1, 2, 3, 5, 6, 7, 8, 9. Solutions posted at
https://learnuw.wisc.edu.

Week |
Tuesday |
Thursday |

1 |
Sept. 3rd Topic: Introduction to the course (slides) and background material on the theory of probability (slides). Readings: Chapter 1,
Appendix (background material on differential/difference
equations and probability). |
Sept. 5th Topic: Discrete Time Markov Chains: description and first properties; examples. Readings: Section 2.1. |

2 |
Sept. 10th HW 1 Due.Topic: Discrete Time Markov Chains: simulation, higher order transition probabilities. Readings: Sections 2.2 -
2.3. |
Sept. 12th. Topic: Discrete time Markov chains: reducibility, periodicity, recurrence and transience. Readings: Section 2.4. |

3 |
Sept.
17th HW 2 Due.Topic: Discrete time Markov chains: reducibility, periodicity, recurrence and transience. Begin stationary distributions. Readings: Sections 2.4 and
2.5. |
Sept. 19th Topic: Discrete time Markov chains: Invariant/stationary distributions. Readings: Section 2.5. |

4 |
Sept. 24th Topic: Invariant distributions. Readings: Section 2.5. |
Sept. 26th Topic: Transient
behavior.Readings: Section 2.6. |

5 |
Oct. 1st HW 3 Due.
Topic: DTMC in biosciences: Genetic models. Readings: Section 3.1. |
Oct.
3rd Class cancelled (made up during
evening exam).Topic: NA. Readings: NA. |

6 |
Oct.
8th HW 4 Due.Topic: DTMC in biosciences: Birth and death processes. Readings: Section 3.3. |
Oct. 10th Topic: DTMC in biosciences: Branching processes. Readings: Section 3.3. |

7 |
Oct.
15th Exam 1
(scheduled for evening) Room = 6203 Social
Sciences.Topic: Tentative; Guest Lecture by Bret
Hanlon (Statistics Department) on Parameter
Estimation for DTMCs. Readings: NA. |
Oct. 17th HW 5 Due.Topic: DTMC in biosciences: Branching processes. Readings: Section 3.3. |

8 |
Oct. 22nd Topic: Finish branching processes. Begin renewal processes. Readings: Sections 3.3 and
4.1. |
Oct. 24th Topic: Renewal processes. Renewal reward processes. Readings: Section 4.1. |

9 |
Oct.
29th HW 6 Due.Topic: Point and Poisson processes. Readings: Section 4.2. |
Oct. 31st Topic: Poisson processes. Readings: Section 4.2. |

10 |
Nov. 5th Topic: Transformations of Poisson processes. Readings: Section 4.2. |
Nov. 7th Topic: Finish transformation of Poisson processes. Begin continuous time Markov chains (CTMCs): basic properties. Readings: Section 4.2. and
Sections 5.1. |

11 |
Nov. 12th Topic: CTMCs: basic properties and explosions. Readings: 5.1 - 5.2. |
Nov. 14th HW 7 Due. Topic: CTMCs: explosions; Forward and backward
equations, generator matrix. Readings: Sections 5.2 - 5.3. |

12 |
Nov. 19th Topic: The generator revisited. Readings: Sections 5.5. |
Nov. 21st
Topic: CTMCs: stationary distributions and limiting behavior. Readings: Section 5.4. |

13 |
Nov. 26th Exam 2 (scheduled for
Monday evening - the 25th) Topic: Stationary distributions. Birth and death processes. Readings: Section 5.4 and
5.6. |
Nov. 28th No class due to Thanksgiving |

14 |
Dec. 3rd Topic: Stochastic models of biochemical processes: general model. 6.1. Readings: |
Dec. 5th Topic: Stochastic models of biochemical processes: general models and simulation. Readings: 6.1 and 6.2. |

15 |
Dec. 10th HW 8 Due.Topic: Stochastic models of biochemical
processes: first order reaction networks, and the law of
large numbers.Readings: 6.3 - 6.4. |
Dec. 12th Topic: The classical scaling and the law of large numbers. Readings: 6.4. |