1.Introduction to Probability Theory

1.1.Introduction

1.2.Sample Space and Events

1.3.Probabilities Defined on Events

1.4.Conditional Probabilities

1.5.Independent Events

1.6.Bayes' Formula

Exercises

References

2.Random Variables

2.1.Random Variables

2.2.Discrete Random Variables

2.3.Continuous Random Variables

2.4.Expectation of a Random Variable

2.5.Jointly Distributed Random Variables

2.6.Moment Generating Functions

2.7.Limit Theorems

2.8.Stochastic Processes

Exercises

References

3.Conditional Probability and Conditional Expectation

3.1.Introduction

3.2.The Discrete Case

3.3.The Continuous Case

3.4.Computing Expectations by Conditioning

3.5.Computing Probabilities by Conditioning

3.6.Some Applications

3.7.An Identity for Compound Random Variables

Exercises

4.Markov Chains

4.1.Introduction

4.2.Chapman-Kolmogorov Equations

4.3.Classification of States

4.4.Limiting Probabilities

4.5.Some Applications

4.6.Mean Time Spent in Transient States

4.7.Branching Processes

4.8.Time Reversible Markov Chains

4.9.Markov Chain Monte Carlo Methods

4.10.Markov Decision Processes

4.11.Hidden Markov Chains

Exercises

References

5.The Exponential Distribution and the Poisson Process

5.1.Introduction

5.2.The Exponential Distribution

5.3.The Poisson Process

5.4.Generalizations of the Poisson Process

Exercises

References

6.Continuous-Time Markov Chains

7.Renewal Theory and Its Applications

8.Queueing Theory

9.Reliability Theory

10.Brownian Motion and Stationary Processes

11.Simulation

Appendix: Solutions to Starred Exercises

Index