Stochastic Processes

Syllabus Details

• Poisson and Renewal Processes: Introduction; Poisson process: homogeneous and non-homogeneous Poisson processes; renewal process, key renewal theorem and some applications.
• Markov Chains: Classification of states, existence of stationary distribution, expected time between successive visits to a state, positive recurrent states; random walk.
• Markov Processes: Continuous time discrete state Markov processes, embedded Markov chains and semi-Markov processes; various birth and death processes, Yule process; the various Markovian queues and their steady state distribution when they exist.
• Martingale Processes: Martingale theorem and some applications, Doob’s inequality.
• Brownian Motion: Brownian motion process as a limit of the random walk, its properties, Brownian bridge.

Course Learning Objective

1. Explain what a stochastic process and a sample are, and discuss the principles and objectives of model building based on Markov chains, Poisson processes and Brownian motion.
2. Discuss the principles and objectives of model building based on Markov chains, Poisson processes, Markov processes and Brownian motion.
3. Determine transition matrices for Markov dependent behavior and summarize process information.
4. Apply the various birth and death models to explain population growth, epidemics and queues.
5. Discuss the notions of long-time behavior including transience, recurrence, and equilibrium in applied situations such as branching processes and random walk.

Recommended Books

1. Ross, S. Stochastic Processes. 2nd ed. John Wiley, 1996
2. Goswami, A. and Rao, B. V. A Course in Applied Stochastic Processes. TRIM, Hindustan Book Agency, 2006