EE6150: Stochastic Modeling and the Theory of Queues
Course information
When: Sep-Dec 2020
Lectures: Slot J
Where: Online
Teaching Assistants: Jaswanthi Mandalapu, Dipayan Sen
Course Content
Poisson processes
Discrete-time Markov chains (DTMCs)
Continuous-time Markov chains (CTMCs)
Queueing models
Introduction to Markov reward processes (if time permits)
Grading
Mid-term: 25%
Final exam: 50%
Assignments: 25%
Textbooks
Vidyadhar Kulkarni, Modeling and analysis of stochastic systems.Third edition, Crc Press, 2016.
Geoffrey Grimmett and David Stirzaker, Probability and random processes, Oxford university press, 2020.
Additional References
Robert Gallager, Stochastic processes: theory for applications. Cambridge University Press, 2013.
Lecture Notes
Schedule
Lecture number | Topics Covered | Section reference (Kulkarni's book) |
---|---|---|
Lecture 1 | Exponential distribution | 5.1 |
Lecture 2 | Poisson processes: Definition Shifted Poisson processes | 5.2 |
Lecture 3 | Poisson processes: Finite dimensional distributions Alternate characterization | 5.2 |
Lecture 4 | Event times in a Poisson process | 5.3 |
Lecture 5 | Superposition and splitting of Poisson processes | 5.4 |
Lecture 6 | Non-homogeneous Poisson processes | 5.5 |
Lecture 7 | Problem solving session | |
Lecture 8 | Discrete time Markov chain (DTMC): Definition, Examples Chapman-Kolmogorov equations | 2.1, 2.2, 2.4 |
Lecture 9 | DTMC: Occupancy times | 2.5 |
Lecture 10 | Computing matrix powers (diagonalizable case) Application to DTMCs First passage time - definition | 2.6, 3.1 |
Lecture 11 | DTMC - First passage times CDF, absorption probabilities | 3.2, 3.3 |
Lecture 12 | DTMC - First passage time Analysis of random walks Expectation of the first passage time | 3.3, 3.4 |
Lecture 13 | DTMC - Expectation of the first passage time | 3.4 |
Lecture 14 | DTMCs: Limiting behavior A study using examples | 4.1 |
Lecture 15 | DTMC - limiting behavior: Classification of states | 4.2 |
Lecture 16 | Problem solving session | |
Lecture 17 | Communicating classes, recurrence and transience | 4.3 |
Lecture 18 | Periodicity | 4.3 |
Lecture 19 | Determining transience/recurrence: infinite DTMCs | 4.4 |
Lecture 20 | Limiting behavior: transient DTMC | 4.5 |
Lecture 21 | Stationary distributions: Construction | Grimmett-Stirzaker - Sec 6.4 |
Lecture 22 | Stationary distributions: Existence | Grimmett-Stirzaker - Sec 6.4 |
Lecture 23 | Convergence to stationary distribution: Aperiodic case | Grimmett-Stirzaker - Sec 6.4 |
Lecture 24 | Convergence to stationary distribution: Cesaro sense | Grimmett-Stirzaker - Sec 6.4 |
Lecture 25 | Examples: limiting behavior | 4.6 |
Lecture 26 | DTMCs with costs | 4.8 |
Lecture 27 | Continuous time Markov chain (CTMC): Definition, alternate characterization | 6.1 |
Lecture 28 | CTMCs: Examples | 6.2 |
Lecture 29 | CTMC: Transient behavior, Chapman-Kolmogorov equationsForward and backward equations | 6.4 |
Lecture 30 | Examples: Applications of forward equations Occupancy times | 6.4, 6.5 |
Lecture 31 | Computation of P(t): Finite/infinite state space | 6.6, 6.7 |
Lecture 32 | Absorption probabilities, First-passage times | 6.8 |
Lecture 33 | CTMCs: Limiting behavior | 6.9-6.11 |