Bayesian Statistics

Bayesian statistics is a theory in the field of statistics based on the Bayesian interpretation of probability, where probability is treated as a degree of belief in a proposition. This approach formally combines prior knowledge about a parameter, expressed as a "prior probability distribution," with evidence from observed data through the application of Bayes' Theorem. The result is a "posterior probability distribution," which represents an updated state of belief about the parameter, effectively providing a framework for learning and revising beliefs in light of new evidence.

1. Foundations of Bayesian Inference
   1. The Bayesian Interpretation of Probability
      1. Probability as a Degree of Belief
         1. Definition and Motivation
         2. Personal Probability Assignments
         3. Coherence and Dutch Book Arguments
      2. Subjective vs. Objective Probability
         1. Subjective Probability
         2. Objective Bayesianism
         3. Reference Priors
         4. Maximum Entropy Principle
         5. Debates and Philosophical Perspectives
      3. Comparison with Frequentist Interpretation
         1. Frequency Interpretation of Probability
         2. Key Differences in Inference
         3. Parameter vs. Data Randomness
         4. Examples Illustrating Contrasts
   2. Bayes' Theorem
      1. Conditional Probability
         1. Definition and Notation
         2. Properties of Conditional Probability
         3. Independence and Conditional Independence
         4. Examples and Applications
      2. The Law of Total Probability
         1. Statement and Derivation
         2. Partition of Sample Space
         3. Applications in Bayesian Inference
      3. Derivation and Formulation of Bayes' Theorem
         1. Mathematical Derivation
         2. General Formulation
         3. Discrete and Continuous Cases
         4. Odds Form of Bayes' Theorem
   3. Components of Bayesian Models
      1. Prior Distribution
         1. Representing Prior Beliefs
            1. Types of Prior Information
            2. Encoding Expert Knowledge
            3. Historical Data as Prior Information
         2. Informative vs. Uninformative Priors
            1. Definition of Informative Priors
            2. Definition of Uninformative Priors
            3. Weakly Informative Priors
            4. Jeffreys Priors
            5. Flat Priors
            6. Improper Priors
         3. Elicitation of Priors
            1. Methods for Eliciting Priors
            2. Expert Elicitation Techniques
            3. Practical Considerations
            4. Prior Predictive Checks
      2. Likelihood
         1. The Role of Data
            1. Likelihood as a Function of Parameters
            2. Data Generating Process
            3. Sampling Models
         2. The Likelihood Principle
            1. Statement and Implications
            2. Relevance to Bayesian Inference
            3. Stopping Rules and Optional Stopping
      3. Posterior Distribution
         1. Updating Beliefs with Data
            1. Bayes' Rule in Practice
            2. Sequential Updating
            3. Learning from Data
         2. The Result of Inference
            1. Interpretation of the Posterior
            2. Posterior as a Summary of Knowledge
            3. Posterior Concentration
      4. Marginal Likelihood
         1. Normalizing Constant
            1. Role in Posterior Calculation
            2. Calculation in Simple Models
            3. Integration Challenges
         2. Role in Model Comparison
            1. Use in Bayes Factors
            2. Model Evidence
            3. Computational Challenges