Supervised Learning

  1. Linear Models
    1. Linear Regression
      1. Simple Linear Regression
        1. Mathematical Foundation
          1. Equation of a Line
            1. Interpretation of Coefficients
              1. Slope Interpretation
                1. Intercept Interpretation
                2. Cost Function
                  1. Mean Squared Error Derivation
                    1. Geometric Interpretation
                    2. Parameter Estimation
                      1. Gradient Descent for Linear Regression
                        1. Analytical Solution
                          1. Normal Equation
                            1. Least Squares Method
                          2. Model Assumptions
                            1. Linearity
                              1. Independence of Errors
                                1. Homoscedasticity
                                  1. Normality of Residuals
                                  2. Residual Analysis
                                    1. Residual Plots
                                      1. Diagnostic Tests
                                    2. Multiple Linear Regression
                                      1. The Model Equation
                                        1. Matrix Formulation
                                          1. Vector Notation
                                          2. Interpretation of Coefficients
                                            1. Partial Effects
                                              1. Holding Other Variables Constant
                                              2. Extended Assumptions
                                                1. No Perfect Multicollinearity
                                                  1. Sufficient Sample Size
                                                  2. Multicollinearity
                                                    1. Detection Methods
                                                      1. Variance Inflation Factor
                                                        1. Treatment Strategies
                                                        2. Model Diagnostics
                                                          1. Residual Analysis
                                                            1. Leverage and Influence
                                                              1. Cook's Distance
                                                              2. Feature Selection in Linear Regression
                                                                1. Forward Selection
                                                                  1. Backward Elimination
                                                                    1. Stepwise Selection
                                                                2. Polynomial Regression
                                                                  1. Concept and Motivation
                                                                    1. Fitting Non-linear Relationships
                                                                      1. Degree of the Polynomial
                                                                        1. Choosing Appropriate Degree
                                                                          1. Bias-Variance Tradeoff
                                                                          2. Feature Expansion
                                                                            1. Polynomial Feature Generation
                                                                              1. Interaction Terms
                                                                              2. Risk of Overfitting
                                                                                1. High-degree Polynomials
                                                                                  1. Validation Strategies
                                                                                  2. Regularization in Polynomial Regression
                                                                                    1. Ridge Regression Application
                                                                                      1. Lasso Regression Application
                                                                                    2. Regularized Linear Models
                                                                                      1. Ridge Regression
                                                                                        1. L2 Regularization Concept
                                                                                          1. Ridge Penalty Term
                                                                                            1. Effect on Coefficients
                                                                                              1. Choosing Regularization Parameter
                                                                                                1. Geometric Interpretation
                                                                                                2. Lasso Regression
                                                                                                  1. L1 Regularization Concept
                                                                                                    1. Lasso Penalty Term
                                                                                                      1. Feature Selection Properties
                                                                                                        1. Sparsity Induction
                                                                                                          1. Coordinate Descent Algorithm
                                                                                                          2. Elastic Net
                                                                                                            1. Combination of L1 and L2 Penalties
                                                                                                              1. Balancing Ridge and Lasso
                                                                                                                1. When to Use Elastic Net
                                                                                                                  1. Parameter Selection
                                                                                                                2. Logistic Regression
                                                                                                                  1. Binary Logistic Regression
                                                                                                                    1. The Sigmoid Function
                                                                                                                      1. Mathematical Definition
                                                                                                                        1. Properties and Shape
                                                                                                                        2. Logistic Model Formulation
                                                                                                                          1. Odds and Log-Odds
                                                                                                                            1. Decision Boundary
                                                                                                                              1. Maximum Likelihood Estimation
                                                                                                                                1. Log Loss
                                                                                                                                  1. Binary Cross-Entropy Derivation
                                                                                                                                    1. Gradient Computation
                                                                                                                                  2. Multiclass Logistic Regression
                                                                                                                                    1. Softmax Function
                                                                                                                                      1. Mathematical Definition
                                                                                                                                        1. Probability Interpretation
                                                                                                                                        2. One-vs-Rest Approach
                                                                                                                                          1. Multinomial Logistic Regression
                                                                                                                                          2. Regularization in Logistic Regression
                                                                                                                                            1. L1 Regularized Logistic Regression
                                                                                                                                              1. L2 Regularized Logistic Regression
                                                                                                                                              2. Interpretation of Coefficients
                                                                                                                                                1. Odds Ratios
                                                                                                                                                  1. Marginal Effects

                                                                                                                                              Previous

                                                                                                                                              2. The Supervised Learning Workflow

                                                                                                                                              Go to top

                                                                                                                                              Next

                                                                                                                                              4. Tree-Based Models