Machine Learning with Scikit-Learn

7.

Model Evaluation and Metrics

7.1.

Data Splitting Strategies

7.1.1.

Train-Test Split

7.1.1.1.

Purpose and Importance

7.1.1.2.

7.1.1.3.

7.1.2.

Stratified Splitting

7.1.2.1.

Preserving Class Distribution

7.1.2.2.

Use in Classification

7.1.3.

Time Series Splitting

7.1.3.1.

Temporal Data Considerations

7.1.3.2.

TimeSeriesSplit

7.1.4.

Group-based Splitting

7.1.4.1.

7.1.4.2.

LeaveOneGroupOut

7.2.

Cross-Validation

7.2.1.

Purpose and Benefits

7.2.2.

K-Fold Cross-Validation

7.2.2.1.

7.2.2.2.

Computational Considerations

7.2.3.

Stratified K-Fold

7.2.3.1.

Class Balance Preservation

7.2.4.

Leave-One-Out Cross-Validation

7.2.4.1.

Extreme Case of K-Fold

7.2.4.2.

Computational Cost

7.2.5.

Leave-P-Out Cross-Validation

7.2.6.

7.2.6.1.

Random Sampling

7.2.7.

Cross-Validation Utilities

7.2.7.1.

cross_val_score

7.2.7.2.

7.2.7.3.

cross_val_predict

7.3.

Classification Metrics

7.3.1.

7.3.1.1.

Definition and Calculation

7.3.1.2.

Limitations with Imbalanced Data

7.3.2.

7.3.2.1.

True Positive Rate

7.3.2.2.

Class-specific Precision

7.3.3.

7.3.3.1.

7.3.3.2.

Class-specific Recall

7.3.4.

7.3.4.1.

7.3.4.2.

Macro and Micro Averaging

7.3.5.

7.3.5.1.

Weighted Harmonic Mean

7.3.6.

Confusion Matrix

7.3.6.1.

7.3.6.2.

False Positives

7.3.6.3.

7.3.6.4.

False Negatives

7.3.6.5.

7.3.7.

Classification Report

7.3.7.1.

Comprehensive Metrics

7.3.8.

7.3.8.1.

True Positive Rate vs False Positive Rate

7.3.8.2.

Threshold Selection

7.3.8.3.

Multi-class Extension

7.3.9.

7.3.9.1.

Area Under the ROC Curve

7.3.9.2.

7.3.10.

Precision-Recall Curve

7.3.10.1.

Imbalanced Dataset Evaluation

7.3.11.

Average Precision Score

7.3.12.

7.3.12.1.

Probabilistic Predictions

7.3.13.

Matthews Correlation Coefficient

7.3.13.1.

Balanced Measure

7.3.14.

7.3.14.1.

Inter-rater Agreement

7.3.15.

7.3.15.1.

Multi-label Classification

7.3.16.

7.3.16.1.

7.4.

Regression Metrics

7.4.1.

Mean Absolute Error

7.4.1.1.

7.4.1.2.

Robustness to Outliers

7.4.2.

Mean Squared Error

7.4.2.1.

7.4.2.2.

Sensitivity to Outliers

7.4.3.

Root Mean Squared Error

7.4.3.1.

Interpretability

7.4.4.

Mean Absolute Percentage Error

7.4.4.1.

7.4.5.

7.4.5.1.

Coefficient of Determination

7.4.5.2.

Explained Variance

7.4.6.

Adjusted R-squared

7.4.6.1.

Penalty for Additional Features

7.4.7.

Mean Squared Log Error

7.4.7.1.

Relative Errors

7.4.8.

Median Absolute Error

7.4.8.1.

7.4.9.

Explained Variance Score

7.5.

Custom Scoring Functions

7.5.1.

Creating Custom Metrics

7.5.2.

make_scorer Function

7.5.3.

Previous

6. Supervised Learning: Classification

Go to top

Next

8. Improving Model Performance