Supervised Learning

  1. Tree-Based Models
    1. Decision Trees
      1. Decision Tree Structure
        1. Nodes and Leaves
          1. Decision Rules
            1. Tree Depth and Width
            2. Decision Tree for Classification
              1. Splitting Criteria
                1. Gini Impurity
                  1. Mathematical Definition
                    1. Calculation Examples
                    2. Information Gain and Entropy
                      1. Entropy Definition
                        1. Information Gain Calculation
                        2. Chi-square Test
                        3. Tree Construction Algorithm
                          1. Recursive Partitioning
                            1. Stopping Criteria
                            2. Handling Categorical Variables
                              1. Feature Importance Calculation
                              2. Decision Tree for Regression
                                1. Splitting Criteria for Regression
                                  1. Variance Reduction
                                    1. Mean Squared Error Reduction
                                    2. Prediction in Regression Trees
                                      1. Leaf Value Determination
                                      2. Tree Pruning
                                        1. Pre-pruning Strategies
                                          1. Maximum Depth
                                            1. Minimum Samples per Leaf
                                              1. Minimum Impurity Decrease
                                              2. Post-pruning Strategies
                                                1. Cost Complexity Pruning
                                                  1. Reduced Error Pruning
                                                2. Advantages and Limitations
                                                  1. Interpretability Benefits
                                                    1. Handling Non-linear Relationships
                                                      1. Overfitting Tendency
                                                        1. Instability Issues
                                                      2. Random Forest
                                                        1. Ensemble Concept
                                                          1. Bootstrap Aggregating
                                                            1. Bootstrap Sampling
                                                              1. Aggregation Strategies
                                                              2. Random Feature Selection
                                                                1. Feature Randomness at Each Split
                                                                  1. Number of Features to Consider
                                                                  2. Forest Construction
                                                                    1. Building Individual Trees
                                                                      1. Combining Predictions
                                                                        1. Majority Voting for Classification
                                                                          1. Averaging for Regression
                                                                        2. Out-of-Bag Evaluation
                                                                          1. OOB Error Estimation
                                                                            1. Variable Importance
                                                                            2. Hyperparameters
                                                                              1. Number of Trees
                                                                                1. Tree Depth
                                                                                  1. Feature Subset Size
                                                                                  2. Advantages over Single Trees
                                                                                    1. Reduced Overfitting
                                                                                      1. Improved Generalization
                                                                                        1. Robustness to Outliers
                                                                                      2. Gradient Boosting Trees
                                                                                        1. Boosting Concept
                                                                                          1. Sequential Learning Process
                                                                                            1. Gradient Boosting Algorithm
                                                                                              1. Additive Model Building
                                                                                                1. Gradient Computation
                                                                                                  1. Weak Learner Training
                                                                                                  2. Loss Functions in Gradient Boosting
                                                                                                    1. Regression Loss Functions
                                                                                                      1. Classification Loss Functions
                                                                                                      2. Regularization Techniques
                                                                                                        1. Learning Rate
                                                                                                          1. Tree Constraints
                                                                                                            1. Subsampling

                                                                                                        Previous

                                                                                                        3. Linear Models

                                                                                                        Go to top

                                                                                                        Next

                                                                                                        5. Instance-Based Learning