Machine Learning Fundamentals

  1. Fundamentals of Model Performance and Improvement
    1. The Bias-Variance Trade-off
      1. Underfitting (High Bias)
        1. Causes and Detection
          1. Model Simplicity
            1. Insufficient Features
              1. Poor Algorithm Choice
              2. Symptoms
                1. Poor Training Performance
                  1. Poor Validation Performance
                    1. Learning Curves Analysis
                  2. Overfitting (High Variance)
                    1. Causes and Detection
                      1. Model Complexity
                        1. Insufficient Data
                          1. Noise Fitting
                          2. Symptoms
                            1. Good Training Performance
                              1. Poor Validation Performance
                                1. Gap in Learning Curves
                              2. Balancing Bias and Variance
                                1. Model Complexity Selection
                                  1. Regularization Techniques
                                    1. Ensemble Methods
                                      1. Cross-Validation Strategies
                                    2. Regularization
                                      1. L1 Regularization (Lasso)
                                        1. Feature Selection Properties
                                          1. Sparse Solutions
                                            1. Automatic Feature Selection
                                            2. Penalty Function
                                              1. Geometric Interpretation
                                              2. L2 Regularization (Ridge)
                                                1. Shrinkage of Coefficients
                                                  1. Coefficient Reduction
                                                    1. Multicollinearity Handling
                                                    2. Penalty Function
                                                      1. Geometric Interpretation
                                                      2. Elastic Net
                                                        1. Combined L1 and L2 Penalties
                                                          1. Mixing Parameter
                                                            1. Best of Both Worlds
                                                            2. Regularization Parameter Selection
                                                              1. Cross-Validation
                                                                1. Information Criteria
                                                                  1. Regularization Paths
                                                                2. Cross-Validation
                                                                  1. The Need for Robust Evaluation
                                                                    1. Single Split Limitations
                                                                      1. Variance Reduction
                                                                        1. Model Selection Reliability
                                                                        2. k-Fold Cross-Validation
                                                                          1. Fold Creation
                                                                            1. Training and Validation Cycles
                                                                              1. Performance Aggregation
                                                                                1. Computational Considerations
                                                                                2. Stratified k-Fold
                                                                                  1. Class Distribution Preservation
                                                                                    1. Imbalanced Data Handling
                                                                                      1. Representative Folds
                                                                                      2. Leave-One-Out Cross-Validation
                                                                                        1. Maximum Data Usage
                                                                                          1. High Variance Estimates
                                                                                            1. Computational Cost
                                                                                            2. Time Series Cross-Validation
                                                                                              1. Temporal Dependencies
                                                                                                1. Forward Chaining
                                                                                                  1. Expanding Windows
                                                                                                  2. Cross-Validation in Model Selection
                                                                                                    1. Hyperparameter Tuning
                                                                                                      1. Algorithm Comparison
                                                                                                        1. Feature Selection

                                                                                                    Previous

                                                                                                    7. Unsupervised Learning: Dimensionality Reduction

                                                                                                    Go to top

                                                                                                    Next

                                                                                                    9. Introduction to Advanced Concepts