Statistics

10.

Correlation and Regression

10.1.

Correlation Analysis

10.1.1.

10.1.1.1.

Measuring Linear Relationships

10.1.1.2.

Strength and Direction

10.1.2.

Pearson Correlation Coefficient (r)

10.1.2.1.

10.1.2.2.

10.1.2.3.

Interpretation of r Values

10.1.2.3.1.

Perfect Correlation

10.1.2.3.2.

Strong Correlation

10.1.2.3.3.

Moderate Correlation

10.1.2.3.4.

Weak Correlation

10.1.2.3.5.

10.1.3.

Properties of Correlation

10.1.3.1.

Unitless Measure

10.1.3.2.

10.1.3.3.

Linear Relationships Only

10.1.4.

Interpreting Correlation

10.1.4.1.

Strength and Direction

10.1.4.2.

Scatter Plot Analysis

10.1.4.2.1.

Linear Patterns

10.1.4.2.2.

Curved Patterns

10.1.4.2.3.

Outlier Effects

10.1.5.

Correlation vs. Causation

10.1.5.1.

Third Variable Problem

10.1.5.2.

Spurious Correlation

10.1.5.3.

Confounding Variables

10.1.5.4.

Establishing Causation

10.1.6.

Other Correlation Measures

10.1.6.1.

Spearman's Rank Correlation

10.1.6.2.

Point-Biserial Correlation

10.2.

Simple Linear Regression

10.2.1.

10.2.1.1.

10.2.1.2.

Modeling Relationships

10.2.2.

The Regression Model

10.2.2.1.

Population Regression Line

10.2.2.2.

Sample Regression Line

10.2.2.3.

Dependent and Independent Variables

10.2.2.4.

10.2.3.

The Method of Least Squares

10.2.3.1.

Minimizing Sum of Squared Errors

10.2.3.2.

Calculation of Slope and Intercept

10.2.3.3.

Normal Equations

10.2.4.

The Equation of the Regression Line

10.2.4.1.

Slope Interpretation

10.2.4.2.

Y-intercept Interpretation

10.2.4.3.

Prediction Equation

10.2.5.

Assumptions of Linear Regression

10.2.5.1.

10.2.5.1.1.

Scatter Plot Examination

10.2.5.1.2.

10.2.5.2.

10.2.5.2.1.

Random Sampling

10.2.5.2.2.

No Autocorrelation

10.2.5.3.

Homoscedasticity

10.2.5.3.1.

Constant Variance

10.2.5.3.2.

Residual Plot Analysis

10.2.5.4.

Normality of Residuals

10.2.5.4.1.

Normal Probability Plots

10.2.5.4.2.

Histogram of Residuals

10.2.6.

Coefficient of Determination (R²)

10.2.6.1.

10.2.6.2.

Explained vs. Unexplained Variation

10.2.6.3.

10.2.7.

Standard Error of the Estimate

10.2.7.1.

10.2.7.2.

10.2.7.3.

Prediction Accuracy

10.2.8.

Residual Analysis

10.2.8.1.

Residual Calculation

10.2.8.2.

10.2.8.3.

Outlier Detection

10.2.8.4.

Influential Points

10.2.9.

Inference about the Slope

10.2.9.1.

Hypothesis Testing

10.2.9.1.1.

Testing Significance

10.2.9.1.2.

t-test for Slope

10.2.9.2.

Confidence Intervals

10.2.9.2.1.

Slope Confidence Interval

10.2.9.2.2.

Mean Response Confidence Interval

10.2.9.2.3.

Prediction Intervals

10.3.

Multiple Linear Regression

10.3.1.

10.3.1.1.

Multiple Predictors

10.3.1.2.

Complex Relationships

10.3.2.

The Multiple Regression Model

10.3.2.1.

Model Specification

10.3.2.2.

Matrix Notation

10.3.2.3.

Interpretation of Coefficients

10.3.2.3.1.

10.3.2.3.2.

Holding Other Variables Constant

10.3.3.

10.3.3.1.

Same as Simple Regression

10.3.3.2.

No Perfect Multicollinearity

10.3.4.

Model Estimation

10.3.4.1.

Least Squares Method

10.3.4.2.

Normal Equations

10.3.5.

Model Evaluation

10.3.5.1.

10.3.5.2.

Adjusted R-squared

10.3.5.2.1.

10.3.5.2.2.

Penalty for Additional Variables

10.3.5.2.3.

10.3.5.3.

F-test for Overall Significance

10.3.6.

Individual Coefficient Testing

10.3.6.1.

t-tests for Each Coefficient

10.3.6.2.

Confidence Intervals

10.3.7.

Multicollinearity

10.3.7.1.

10.3.7.1.1.

Correlation Matrix

10.3.7.1.2.

Variance Inflation Factor (VIF)

10.3.7.1.3.

10.3.7.2.

10.3.7.3.

10.3.7.3.1.

Variable Selection

10.3.7.3.2.

Ridge Regression

10.3.8.

Model Building and Selection

10.3.8.1.

Forward Selection

10.3.8.2.

Backward Elimination

10.3.8.3.

Stepwise Regression

10.3.8.4.

Model Comparison Criteria

10.3.8.4.1.

AIC (Akaike Information Criterion)

10.3.8.4.2.

BIC (Bayesian Information Criterion)

10.3.8.4.3.

Adjusted R-squared

Previous

9. Analysis of Variance (ANOVA)

Go to top

Next

11. Chi-Square Tests