Python for Data Science

6.

Introduction to Machine Learning with Scikit-learn

6.1.

Machine Learning Fundamentals

6.1.1.

6.1.1.1.

What is Machine Learning

6.1.1.2.

Types of Machine Learning Problems

6.1.1.2.1.

Supervised Learning

6.1.1.2.1.1.

Classification Problems

6.1.1.2.1.2.

Regression Problems

6.1.1.2.2.

Unsupervised Learning

6.1.1.2.2.1.

6.1.1.2.2.2.

Dimensionality Reduction

6.1.1.2.2.3.

Association Rules

6.1.1.2.3.

Semi-supervised Learning

6.1.1.2.4.

Reinforcement Learning

6.1.1.3.

Machine Learning Workflow

6.1.1.3.1.

Problem Definition

6.1.1.3.2.

Data Collection and Exploration

6.1.1.3.3.

Data Preprocessing

6.1.1.3.4.

Model Selection

6.1.1.3.5.

Training and Validation

6.1.1.3.6.

Model Evaluation

6.1.1.3.7.

Deployment and Monitoring

6.1.2.

Key Terminology

6.1.2.1.

Features and Target Variables

6.1.2.2.

Training and Test Sets

6.1.2.3.

Overfitting and Underfitting

6.1.2.4.

Bias-Variance Tradeoff

6.1.2.5.

Cross-validation

6.1.2.6.

Hyperparameters

6.1.3.

Scikit-learn Ecosystem

6.1.3.1.

Library Overview

6.1.3.2.

Integration with NumPy and Pandas

6.1.3.3.

Consistent API Design

6.1.3.4.

Community and Documentation

6.2.

Scikit-learn API and Design Patterns

6.2.1.

Core API Components

6.2.1.1.

Estimator Objects

6.2.1.1.1.

Estimator Interface

6.2.1.1.2.

Parameter Setting

6.2.1.1.3.

State Management

6.2.1.2.

Predictor Interface

6.2.1.2.1.

predict() Method

6.2.1.2.2.

predict_proba() Method

6.2.1.2.3.

decision_function() Method

6.2.1.3.

Transformer Interface

6.2.1.3.1.

6.2.1.3.2.

transform() Method

6.2.1.3.3.

fit_transform() Method

6.2.2.

API Consistency

6.2.2.1.

Method Naming Conventions

6.2.2.2.

Parameter Conventions

6.2.2.3.

Return Value Patterns

6.2.3.

Estimator Types

6.2.3.1.

6.2.3.2.

6.2.3.3.

6.2.3.4.

6.3.

Data Preprocessing and Feature Engineering

6.3.1.

Data Preparation Workflow

6.3.1.1.

Data Quality Assessment

6.3.1.2.

Missing Data Strategies

6.3.1.3.

Outlier Detection and Treatment

6.3.2.

Feature Scaling and Normalization

6.3.2.1.

Why Scaling Matters

6.3.2.2.

Standardization

6.3.2.2.1.

6.3.2.2.2.

Z-score Normalization

6.3.2.2.3.

6.3.2.3.

6.3.2.3.1.

6.3.2.3.2.

6.3.2.3.3.

Unit Vector Scaling

6.3.2.4.

When to Apply Scaling

6.3.3.

Categorical Data Encoding

6.3.3.1.

6.3.3.1.1.

6.3.3.1.2.

Ordinal Encoding

6.3.3.2.

One-Hot Encoding

6.3.3.2.1.

6.3.3.2.2.

get_dummies() Alternative

6.3.3.2.3.

Handling High Cardinality

6.3.3.3.

Target Encoding

6.3.3.4.

Binary Encoding

6.3.4.

Feature Selection

6.3.4.1.

6.3.4.1.1.

Statistical Tests

6.3.4.1.2.

Correlation Analysis

6.3.4.2.

Wrapper Methods

6.3.4.2.1.

Recursive Feature Elimination

6.3.4.2.2.

Forward/Backward Selection

6.3.4.3.

Embedded Methods

6.3.4.3.1.

L1 Regularization

6.3.4.3.2.

Tree-based Feature Importance

6.3.5.

Feature Creation

6.3.5.1.

Polynomial Features

6.3.5.2.

Interaction Terms

6.3.5.3.

Domain-specific Features

6.3.6.

6.3.6.1.

Train-Test Split

6.3.6.1.1.

train_test_split() Function

6.3.6.1.2.

6.3.6.1.3.

6.3.6.2.

Train-Validation-Test Split

6.3.6.3.

Time Series Splitting

6.3.7.

Pipeline Construction

6.3.7.1.

Pipeline Concept

6.3.7.2.

Creating Pipelines

6.3.7.2.1.

6.3.7.2.2.

make_pipeline() Function

6.3.7.3.

Pipeline Benefits

6.3.7.3.1.

Code Organization

6.3.7.3.2.

Parameter Tuning

6.3.7.3.3.

Avoiding Data Leakage

6.3.7.4.

Column Transformers

6.3.7.4.1.

ColumnTransformer Class

6.3.7.4.2.

Heterogeneous Data Processing

6.4.

Supervised Learning Algorithms

6.4.1.

6.4.1.1.

Linear Regression

6.4.1.1.1.

Ordinary Least Squares

6.4.1.1.2.

Assumptions and Limitations

6.4.1.1.3.

Coefficient Interpretation

6.4.1.2.

Regularized Linear Models

6.4.1.2.1.

Ridge Regression

6.4.1.2.1.1.

L2 Regularization

6.4.1.2.1.2.

Alpha Parameter

6.4.1.2.2.

Lasso Regression

6.4.1.2.2.1.

L1 Regularization

6.4.1.2.2.2.

Feature Selection Properties

6.4.1.2.3.

6.4.1.2.3.1.

Combined L1 and L2

6.4.1.2.3.2.

l1_ratio Parameter

6.4.1.3.

Logistic Regression

6.4.1.3.1.

Binary Classification

6.4.1.3.2.

Multiclass Extensions

6.4.1.3.3.

Probability Interpretation

6.4.1.3.4.

Regularization Options

6.4.2.

Tree-Based Models

6.4.2.1.

6.4.2.1.1.

Tree Construction Algorithm

6.4.2.1.2.

Splitting Criteria

6.4.2.1.2.1.

6.4.2.1.2.2.

6.4.2.1.2.3.

Mean Squared Error

6.4.2.1.3.

6.4.2.1.4.

Hyperparameters

6.4.2.1.4.1.

6.4.2.1.4.2.

min_samples_split

6.4.2.1.4.3.

min_samples_leaf

6.4.2.2.

Ensemble Methods

6.4.2.2.1.

6.4.2.2.1.1.

Bootstrap Aggregating

6.4.2.2.1.2.

Feature Randomness

6.4.2.2.1.3.

Out-of-bag Error

6.4.2.2.1.4.

Feature Importance

6.4.2.2.2.

Gradient Boosting

6.4.2.2.2.1.

Boosting Concept

6.4.2.2.2.2.

GradientBoostingClassifier

6.4.2.2.2.3.

GradientBoostingRegressor

6.4.2.2.2.4.

Learning Rate and Estimators

6.4.2.2.3.

6.4.2.2.3.1.

Extremely Randomized Trees

6.4.2.2.3.2.

Differences from Random Forest

6.4.3.

Support Vector Machines

6.4.3.1.

6.4.3.1.1.

Maximum Margin Principle

6.4.3.1.2.

Support Vectors

6.4.3.1.3.

6.4.3.2.

SVM for Classification

6.4.3.2.1.

6.4.3.2.2.

6.4.3.2.3.

Kernel Functions

6.4.3.2.3.1.

6.4.3.2.3.2.

Polynomial Kernel

6.4.3.2.3.3.

6.4.3.3.

SVM for Regression

6.4.3.3.1.

Support Vector Regression

6.4.3.3.2.

Epsilon Parameter

6.4.3.4.

SVM Hyperparameters

6.4.3.4.1.

6.4.3.4.2.

Gamma Parameter

6.4.3.4.3.

Kernel Selection

6.4.4.

Instance-Based Learning

6.4.4.1.

K-Nearest Neighbors

6.4.4.1.1.

6.4.4.1.2.

Distance Metrics

6.4.4.1.2.1.

Euclidean Distance

6.4.4.1.2.2.

Manhattan Distance

6.4.4.1.2.3.

Minkowski Distance

6.4.4.1.3.

KNN for Classification

6.4.4.1.4.

KNN for Regression

6.4.4.1.5.

6.4.4.1.6.

Curse of Dimensionality

6.4.5.

6.4.5.1.

6.4.5.2.

Naive Assumption

6.4.5.3.

Gaussian Naive Bayes

6.4.5.4.

Multinomial Naive Bayes

6.4.5.5.

Bernoulli Naive Bayes

6.5.

Unsupervised Learning Algorithms

6.5.1.

Clustering Algorithms

6.5.1.1.

K-Means Clustering

6.5.1.1.1.

Algorithm Steps

6.5.1.1.2.

Centroid Initialization

6.5.1.1.3.

Choosing Number of Clusters

6.5.1.1.3.1.

6.5.1.1.3.2.

Silhouette Analysis

6.5.1.1.4.

K-Means Limitations

6.5.1.2.

Hierarchical Clustering

6.5.1.2.1.

Agglomerative Clustering

6.5.1.2.2.

Linkage Criteria

6.5.1.2.2.1.

6.5.1.2.2.2.

Complete Linkage

6.5.1.2.2.3.

Average Linkage

6.5.1.2.2.4.

6.5.1.2.3.

Dendrogram Interpretation

6.5.1.3.

Density-Based Clustering

6.5.1.3.1.

DBSCAN Algorithm

6.5.1.3.2.

Core Points and Noise

6.5.1.3.3.

Epsilon and MinPts Parameters

6.5.1.3.4.

Handling Irregular Shapes

6.5.1.4.

Other Clustering Methods

6.5.1.4.1.

6.5.1.4.2.

Spectral Clustering

6.5.1.4.3.

Gaussian Mixture Models

6.5.2.

Dimensionality Reduction

6.5.2.1.

Principal Component Analysis (PCA)

6.5.2.1.1.

6.5.2.1.2.

Eigenvalues and Eigenvectors

6.5.2.1.3.

Explained Variance

6.5.2.1.4.

Number of Components Selection

6.5.2.1.5.

PCA for Visualization

6.5.2.1.6.

PCA Limitations

6.5.2.2.

Other Dimensionality Reduction Techniques

6.5.2.2.1.

Linear Discriminant Analysis (LDA)

6.5.2.2.2.

6.5.2.2.3.

6.5.2.2.4.

Factor Analysis

6.6.

Model Evaluation and Selection

6.6.1.

Evaluation Metrics

6.6.1.1.

Regression Metrics

6.6.1.1.1.

Mean Absolute Error (MAE)

6.6.1.1.2.

Mean Squared Error (MSE)

6.6.1.1.3.

Root Mean Squared Error (RMSE)

6.6.1.1.4.

R-squared (Coefficient of Determination)

6.6.1.1.5.

Adjusted R-squared

6.6.1.1.6.

Mean Absolute Percentage Error (MAPE)

6.6.1.2.

Classification Metrics

6.6.1.2.1.

6.6.1.2.2.

6.6.1.2.3.

Recall (Sensitivity)

6.6.1.2.4.

6.6.1.2.5.

6.6.1.2.6.

Confusion Matrix

6.6.1.2.6.1.

True/False Positives/Negatives

6.6.1.2.6.2.

Matrix Interpretation

6.6.1.2.7.

6.6.1.2.7.1.

True Positive Rate

6.6.1.2.7.2.

False Positive Rate

6.6.1.2.7.3.

AUC (Area Under Curve)

6.6.1.2.8.

Precision-Recall Curve

6.6.1.2.9.

Multi-class Metrics

6.6.1.2.9.1.

Macro Averaging

6.6.1.2.9.2.

Micro Averaging

6.6.1.2.9.3.

Weighted Averaging

6.6.2.

Cross-Validation

6.6.2.1.

Cross-Validation Concept

6.6.2.2.

K-Fold Cross-Validation

6.6.2.2.1.

6.6.2.2.2.

Stratified K-Fold

6.6.2.2.3.

Repeated K-Fold

6.6.2.3.

Leave-One-Out Cross-Validation

6.6.2.4.

Time Series Cross-Validation

6.6.2.5.

Cross-Validation Scoring

6.6.3.

Model Selection Strategies

6.6.3.1.

Hyperparameter Tuning

6.6.3.1.1.

6.6.3.1.1.1.

6.6.3.1.1.2.

Parameter Grids

6.6.3.1.1.3.

Exhaustive Search

6.6.3.1.2.

6.6.3.1.2.1.

RandomizedSearchCV

6.6.3.1.2.2.

Sampling Distributions

6.6.3.1.2.3.

Efficiency Benefits

6.6.3.1.3.

Bayesian Optimization

6.6.3.2.

Model Comparison

6.6.3.2.1.

Cross-Validation Scores

6.6.3.2.2.

Statistical Significance

6.6.3.2.3.

Learning Curves

6.6.3.2.4.

Validation Curves

6.6.4.

Model Diagnostics

6.6.4.1.

Overfitting Detection

6.6.4.1.1.

Training vs Validation Performance

6.6.4.1.2.

Learning Curves

6.6.4.2.

Underfitting Detection

6.6.4.2.1.

Model Complexity Analysis

6.6.4.3.

Bias-Variance Analysis

6.6.4.4.

Residual Analysis

6.6.4.4.1.

6.6.4.4.2.

Normality Tests

6.6.4.4.3.

Homoscedasticity

Previous

5. Data Visualization

Go to top

Next

7. Advanced and Specialized Topics