Stochastic Processes

  1. Continuous-Time Markov Chains
    1. Basic Framework
      1. State Space and Time Parameter
        1. Transition Functions
          1. Markov Property in Continuous Time
            1. Right-Continuity of Sample Paths
            2. Construction and Properties
              1. Holding Times
                1. Exponential Distribution
                  1. Memoryless Property
                    1. Rate Parameters
                    2. Jump Chain
                      1. Embedded Discrete-Time Chain
                        1. Jump Probabilities
                        2. Generator Matrix
                          1. Definition and Properties
                            1. Q-Matrix Structure
                              1. Relationship to Transition Rates
                            2. Kolmogorov Equations
                              1. Forward Equations
                                1. Derivation
                                  1. Matrix Form
                                  2. Backward Equations
                                    1. Derivation
                                      1. Matrix Form
                                      2. Solutions and Uniqueness
                                      3. Classification of States
                                        1. Communicating Classes
                                          1. Recurrence and Transience
                                            1. Positive and Null Recurrence
                                              1. Irreducibility
                                              2. Long-Run Behavior
                                                1. Limiting Probabilities
                                                  1. Stationary Distribution
                                                    1. Balance Equations
                                                      1. Global Balance
                                                        1. Detailed Balance
                                                        2. Ergodic Properties
                                                        3. Birth-and-Death Processes
                                                          1. Definition and Structure
                                                            1. Birth Rates and Death Rates
                                                              1. Generator Matrix Structure
                                                                1. Equilibrium Distribution
                                                                  1. Product Form Solution
                                                                    1. Normalization
                                                                    2. Special Cases
                                                                      1. Pure Birth Process
                                                                        1. Pure Death Process
                                                                          1. Linear Birth-Death Process
                                                                        2. Queueing Applications
                                                                          1. M/M/1 Queue
                                                                            1. Model Description
                                                                              1. Steady-State Analysis
                                                                                1. Performance Measures
                                                                                2. M/M/s Queue
                                                                                  1. Multiple Servers
                                                                                    1. Steady-State Distribution
                                                                                      1. Blocking Probabilities
                                                                                      2. M/M/∞ Queue
                                                                                        1. M/M/s/K Queue
                                                                                        2. Time Reversibility
                                                                                          1. Detailed Balance Conditions
                                                                                            1. Reversible Processes

                                                                                          Previous

                                                                                          4. Poisson Processes

                                                                                          Go to top

                                                                                          Next

                                                                                          6. Renewal Theory