by K Borovkov (The University of Melbourne, Australia)

This textbook has been developed from the lecture notes for a one-semester course on stochastic modelling. It reviews the basics of probability theory and then covers the following topics: Markov chains, Markov decision processes, jump Markov processes, elements of queueing theory, basic renewal theory, elements of time series and simulation. Rigorous proofs are often replaced with sketches of arguments — with indications as to why a particular result holds, and also how it is connected with other results — and illustrated by examples. Wherever possible, the book includes references to more specialised texts containing both proofs and more advanced material related to the topics covered.

• Basics of Probability Theory
• Markov Chains
• Markov Decision Processes
• The Exponential Distribution and Poisson Process
• Jump Markov Processes
• Elements of Queueing Theory
• Elements of Renewal Theory
• Elements of Time Series
• Elements of Simulation