UsefulLinks
Systems Science
Operations Research
1. Introduction to Operations Research
2. Mathematical Preliminaries
3. Linear Programming
4. Transportation and Assignment Problems
5. Network Optimization Models
6. Integer Programming
7. Nonlinear Programming
8. Dynamic Programming
9. Stochastic Processes and Markov Chains
10. Queueing Theory
11. Inventory Theory
12. Simulation
13. Decision Analysis
14. Heuristics and Metaheuristics
9.
Stochastic Processes and Markov Chains
9.1.
Introduction to Stochastic Processes
9.1.1.
Classification of Stochastic Processes
9.1.2.
Discrete vs. Continuous Time
9.1.3.
Discrete vs. Continuous State Space
9.2.
Markov Chains
9.2.1.
Markov Property
9.2.2.
State Space and Transition Probabilities
9.2.3.
Transition Probability Matrix
9.2.4.
Chapman-Kolmogorov Equations
9.2.5.
N-Step Transition Probabilities
9.3.
Classification of States
9.3.1.
Accessible and Communicating States
9.3.2.
Recurrent and Transient States
9.3.3.
Periodic and Aperiodic States
9.3.4.
Irreducible Chains
9.4.
Long-Run Behavior
9.4.1.
Steady-State Probabilities
9.4.2.
Limiting Distributions
9.4.3.
Ergodic Theorems
9.5.
Absorbing Markov Chains
9.5.1.
Absorbing States
9.5.2.
Fundamental Matrix
9.5.3.
Absorption Probabilities
9.5.4.
Expected Time to Absorption
9.6.
Applications of Markov Chains
9.6.1.
Reliability Analysis
9.6.2.
Inventory Systems
9.6.3.
Queueing Systems
9.6.4.
Brand Switching Models
Previous
8. Dynamic Programming
Go to top
Next
10. Queueing Theory