Stochastic Processes

5.

Continuous-Time Markov Chains

5.1.

Basic Framework

5.1.1.

State Space and Time Parameter

5.1.2.

Transition Functions

5.1.3.

Markov Property in Continuous Time

5.1.4.

Right-Continuity of Sample Paths

5.2.

Construction and Properties

5.2.1.

5.2.1.1.

Exponential Distribution

5.2.1.2.

Memoryless Property

5.2.1.3.

Rate Parameters

5.2.2.

5.2.2.1.

Embedded Discrete-Time Chain

5.2.2.2.

Jump Probabilities

5.2.3.

Generator Matrix

5.2.3.1.

Definition and Properties

5.2.3.2.

Q-Matrix Structure

5.2.3.3.

Relationship to Transition Rates

5.3.

Kolmogorov Equations

5.3.1.

Forward Equations

5.3.1.1.

5.3.1.2.

5.3.2.

Backward Equations

5.3.2.1.

5.3.2.2.

5.3.3.

Solutions and Uniqueness

5.4.

Classification of States

5.4.1.

Communicating Classes

5.4.2.

Recurrence and Transience

5.4.3.

Positive and Null Recurrence

5.4.4.

5.5.

Long-Run Behavior

5.5.1.

Limiting Probabilities

5.5.2.

Stationary Distribution

5.5.2.1.

Balance Equations

5.5.2.2.

5.5.2.3.

Detailed Balance

5.5.3.

Ergodic Properties

5.6.

Birth-and-Death Processes

5.6.1.

Definition and Structure

5.6.2.

Birth Rates and Death Rates

5.6.3.

Generator Matrix Structure

5.6.4.

Equilibrium Distribution

5.6.4.1.

Product Form Solution

5.6.4.2.

5.6.5.

5.6.5.1.

Pure Birth Process

5.6.5.2.

Pure Death Process

5.6.5.3.

Linear Birth-Death Process

5.7.

Queueing Applications

5.7.1.

5.7.1.1.

Model Description

5.7.1.2.

Steady-State Analysis

5.7.1.3.

Performance Measures

5.7.2.

5.7.2.1.

Multiple Servers

5.7.2.2.

Steady-State Distribution

5.7.2.3.

Blocking Probabilities

5.7.3.

5.7.4.

5.8.

Time Reversibility

5.8.1.

Detailed Balance Conditions

5.8.2.

Reversible Processes

Previous

4. Poisson Processes

Go to top

Next

6. Renewal Theory