Data Science

7.

Machine Learning Fundamentals

7.1.

7.1.1.

What is Machine Learning

7.1.1.1.

Definition and Scope

7.1.1.2.

AI vs Machine Learning vs Deep Learning

7.1.1.3.

Applications and Use Cases

7.1.2.

Types of Machine Learning

7.1.2.1.

Supervised Learning

7.1.2.1.1.

7.1.2.1.2.

7.1.2.2.

Unsupervised Learning

7.1.2.2.1.

7.1.2.2.2.

Dimensionality Reduction

7.1.2.2.3.

Association Rules

7.1.2.3.

Semi-supervised Learning

7.1.2.4.

Reinforcement Learning

7.1.2.5.

Online Learning

7.1.2.6.

Transfer Learning

7.1.3.

The Machine Learning Workflow

7.1.3.1.

Problem Definition

7.1.3.2.

Data Collection

7.1.3.3.

Data Preprocessing

7.1.3.4.

Feature Engineering

7.1.3.5.

Model Selection

7.1.3.6.

7.1.3.7.

7.1.3.8.

7.1.3.9.

7.1.4.

Training and Testing

7.1.4.1.

7.1.4.2.

7.1.4.3.

7.1.4.4.

Cross-validation

7.1.4.5.

Hold-out Validation

7.1.5.

Overfitting and Underfitting

7.1.5.1.

7.1.5.2.

7.1.5.3.

Detection Methods

7.1.5.4.

Prevention Strategies

7.1.6.

Bias-Variance Tradeoff

7.1.6.1.

Bias Definition

7.1.6.2.

Variance Definition

7.1.6.3.

Tradeoff Implications

7.1.6.4.

Model Complexity Effects

7.1.7.

No Free Lunch Theorem

7.1.8.

Curse of Dimensionality

7.2.

Supervised Learning - Regression

7.2.1.

Linear Regression

7.2.1.1.

Simple Linear Regression

7.2.1.1.1.

Mathematical Foundation

7.2.1.1.2.

Least Squares Method

7.2.1.1.3.

7.2.1.1.4.

7.2.1.2.

Multiple Linear Regression

7.2.1.2.1.

Matrix Formulation

7.2.1.2.2.

Parameter Estimation

7.2.1.2.3.

Statistical Inference

7.2.1.2.4.

Model Diagnostics

7.2.1.3.

Polynomial Regression

7.2.1.3.1.

Polynomial Features

7.2.1.3.2.

Degree Selection

7.2.1.3.3.

Overfitting Concerns

7.2.2.

Regularized Regression

7.2.2.1.

Ridge Regression

7.2.2.1.1.

L2 Regularization

7.2.2.1.2.

Hyperparameter Tuning

7.2.2.1.3.

Geometric Interpretation

7.2.2.2.

Lasso Regression

7.2.2.2.1.

L1 Regularization

7.2.2.2.2.

Feature Selection Properties

7.2.2.2.3.

Coordinate Descent

7.2.2.3.

7.2.2.3.1.

Combined L1 and L2

7.2.2.3.2.

Parameter Selection

7.2.3.

Non-linear Regression

7.2.3.1.

Support Vector Regression

7.2.3.1.1.

7.2.3.1.2.

Hyperparameter Tuning

7.2.3.1.3.

Advantages and Disadvantages

7.2.3.2.

Decision Tree Regression

7.2.3.2.1.

Tree Construction

7.2.3.2.2.

Splitting Criteria

7.2.3.2.3.

Pruning Techniques

7.2.3.3.

Ensemble Methods

7.2.3.3.1.

Random Forest Regression

7.2.3.3.2.

Gradient Boosting Regression

7.2.3.3.3.

7.2.3.3.4.

7.2.4.

Regression Evaluation

7.2.4.1.

Mean Absolute Error

7.2.4.2.

Mean Squared Error

7.2.4.3.

Root Mean Squared Error

7.2.4.4.

7.2.4.5.

Adjusted R-squared

7.2.4.6.

Mean Absolute Percentage Error

7.3.

Supervised Learning - Classification

7.3.1.

Linear Classification

7.3.1.1.

Logistic Regression

7.3.1.1.1.

Sigmoid Function

7.3.1.1.2.

Maximum Likelihood Estimation

7.3.1.1.3.

Multiclass Extensions

7.3.1.1.4.

7.3.1.2.

Linear Discriminant Analysis

7.3.1.2.1.

7.3.1.2.2.

Decision Boundaries

7.3.1.2.3.

Quadratic Discriminant Analysis

7.3.2.

Instance-based Learning

7.3.2.1.

k-Nearest Neighbors

7.3.2.1.1.

Distance Metrics

7.3.2.1.2.

7.3.2.1.3.

Weighted Voting

7.3.2.1.4.

Curse of Dimensionality

7.3.3.

Probabilistic Classifiers

7.3.3.1.

7.3.3.1.1.

Bayes' Theorem Foundation

7.3.3.1.2.

Independence Assumption

7.3.3.1.3.

Gaussian Naive Bayes

7.3.3.1.4.

Multinomial Naive Bayes

7.3.3.1.5.

Bernoulli Naive Bayes

7.3.4.

Support Vector Machines

7.3.4.1.

7.3.4.1.1.

Maximum Margin Principle

7.3.4.1.2.

Support Vectors

7.3.4.1.3.

7.3.4.2.

7.3.4.2.1.

Kernel Functions

7.3.4.2.2.

7.3.4.2.3.

Polynomial Kernel

7.3.4.3.

Hyperparameter Tuning

7.3.5.

Tree-based Methods

7.3.5.1.

7.3.5.1.1.

Splitting Criteria

7.3.5.1.2.

Information Gain

7.3.5.1.3.

7.3.5.1.4.

7.3.5.2.

Ensemble Methods

7.3.5.2.1.

7.3.5.2.2.

Gradient Boosting

7.3.5.2.3.

7.3.5.2.4.

7.3.5.2.5.

7.3.6.

Neural Networks

7.3.6.1.

7.3.6.2.

Multi-layer Perceptron

7.3.6.3.

Backpropagation

7.3.6.4.

Activation Functions

7.3.7.

Classification Evaluation

7.3.7.1.

7.3.7.2.

7.3.7.3.

7.3.7.4.

7.3.7.5.

Confusion Matrix

7.3.7.6.

7.3.7.7.

7.3.7.8.

Precision-Recall Curve

7.3.7.9.

Classification Report

7.4.

Unsupervised Learning

7.4.1.

7.4.1.1.

k-Means Clustering

7.4.1.1.1.

Algorithm Steps

7.4.1.1.2.

Initialization Methods

7.4.1.1.3.

7.4.1.1.4.

7.4.1.2.

Hierarchical Clustering

7.4.1.2.1.

Agglomerative Clustering

7.4.1.2.2.

Divisive Clustering

7.4.1.2.3.

Linkage Criteria

7.4.1.2.4.

7.4.1.3.

Density-based Clustering

7.4.1.3.1.

7.4.1.3.2.

7.4.1.3.3.

7.4.1.4.

Model-based Clustering

7.4.1.4.1.

Gaussian Mixture Models

7.4.1.4.2.

Expectation-Maximization Algorithm

7.4.1.5.

Clustering Evaluation

7.4.1.5.1.

Silhouette Score

7.4.1.5.2.

Davies-Bouldin Index

7.4.1.5.3.

Calinski-Harabasz Index

7.4.1.5.4.

Adjusted Rand Index

7.4.2.

Dimensionality Reduction

7.4.2.1.

Principal Component Analysis

7.4.2.1.1.

Mathematical Foundation

7.4.2.1.2.

Eigenvalue Decomposition

7.4.2.1.3.

Explained Variance

7.4.2.1.4.

Component Interpretation

7.4.2.2.

Factor Analysis

7.4.2.2.1.

Latent Variables

7.4.2.2.2.

Factor Loadings

7.4.2.2.3.

Rotation Methods

7.4.2.3.

Independent Component Analysis

7.4.2.3.1.

Non-Gaussian Assumption

7.4.2.4.

Non-linear Methods

7.4.2.4.1.

7.4.2.4.2.

7.4.2.4.3.

7.4.2.4.4.

Locally Linear Embedding

7.4.3.

Association Rule Mining

7.4.3.1.

Market Basket Analysis

7.4.3.2.

Apriori Algorithm

7.4.3.3.

FP-Growth Algorithm

7.4.3.4.

Support and Confidence

7.4.3.5.

Lift and Conviction

7.5.

Model Selection and Evaluation

7.5.1.

Cross-validation Techniques

7.5.1.1.

k-Fold Cross-validation

7.5.1.2.

Stratified k-Fold

7.5.1.3.

Leave-One-Out Cross-validation

7.5.1.4.

Time Series Cross-validation

7.5.1.5.

Nested Cross-validation

7.5.2.

Hyperparameter Optimization

7.5.2.1.

7.5.2.2.

7.5.2.3.

Bayesian Optimization

7.5.2.4.

7.5.2.5.

Population-based Training

7.5.3.

Model Comparison

7.5.3.1.

Statistical Significance Testing

7.5.3.2.

7.5.3.3.

7.5.3.4.

Wilcoxon Signed-rank Test

7.5.4.

Learning Curves

7.5.4.1.

Training vs Validation Error

7.5.4.2.

Sample Size Effects

7.5.4.3.

Diagnosing Bias and Variance

7.5.5.

Feature Importance

7.5.5.1.

Permutation Importance

7.5.5.2.

Tree-based Importance

7.5.5.3.

Coefficient Analysis

7.5.5.4.

Previous

6. Feature Engineering and Selection

Go to top

Next

8. Advanced Machine Learning Topics