Regression Analysis

2.

Simple Linear Regression

2.1.

The Simple Linear Regression Model

2.1.1.

Mathematical Formulation

2.1.2.

Graphical Representation

2.1.3.

Components of the Model

2.1.4.

Assumptions of the Model

2.2.

Method of Ordinary Least Squares

2.2.1.

Objective Function of OLS

2.2.2.

Minimizing Sum of Squared Residuals

2.2.3.

Derivation of OLS Estimators

2.2.3.1.

First-Order Conditions

2.2.3.2.

Normal Equations

2.2.3.3.

Estimator for the Intercept

2.2.3.4.

Estimator for the Slope

2.2.4.

Geometric Interpretation of OLS

2.3.

Properties of OLS Estimators

2.3.1.

2.3.1.1.

Mathematical Proof

2.3.1.2.

Conditions for Unbiasedness

2.3.2.

2.3.2.1.

Minimum Variance Property

2.3.2.2.

Comparison with Other Estimators

2.3.3.

2.3.3.1.

Large Sample Properties

2.3.3.2.

Convergence in Probability

2.4.

Classical Linear Regression Model Assumptions

2.4.1.

Linearity in Parameters

2.4.2.

Random Sampling

2.4.3.

Sample Variation in Explanatory Variable

2.4.4.

Zero Conditional Mean of Error Term

2.4.5.

Homoscedasticity

2.4.6.

No Serial Correlation

2.4.7.

Normality of Error Terms

2.4.7.1.

Role in Statistical Inference

2.4.7.2.

Central Limit Theorem Applications

2.5.

Decomposition of Variation

2.5.1.

Total Sum of Squares

2.5.2.

Explained Sum of Squares

2.5.3.

Residual Sum of Squares

2.5.4.

Relationship Among Components

2.5.5.

Degrees of Freedom

2.6.

Goodness-of-Fit Measures

2.6.1.

Coefficient of Determination

2.6.1.1.

Calculation and Formula

2.6.1.2.

Interpretation and Meaning

2.6.1.3.

Limitations and Misinterpretations

2.6.2.

Standard Error of Regression

2.6.2.1.

Calculation and Formula

2.6.2.2.

Interpretation and Uses

2.6.3.

Correlation Coefficient

2.7.

Statistical Inference in Simple Linear Regression

2.7.1.

Sampling Distributions of OLS Estimators

2.7.1.1.

Distribution of Slope Estimator

2.7.1.2.

Distribution of Intercept Estimator

2.7.2.

Hypothesis Testing for Coefficients

2.7.2.1.

Null and Alternative Hypotheses

2.7.2.2.

Test Statistics

2.7.2.3.

2.7.2.4.

Critical Values and Decision Rules

2.7.2.5.

P-values and Interpretation

2.7.3.

Confidence Intervals

2.7.3.1.

Construction of Confidence Intervals

2.7.3.2.

Interpretation of Confidence Intervals

2.7.3.3.

Confidence Intervals for Predictions

2.7.4.

Practical vs Statistical Significance

2.8.

Prediction and Forecasting

2.8.1.

Point Predictions

2.8.2.

Prediction Intervals

2.8.3.

Confidence Intervals for Mean Response

2.8.4.

Out-of-Sample Prediction

Previous

1. Foundations of Regression Analysis

Go to top

Next

3. Multiple Linear Regression