Feature Engineering for Machine Learning

11.

Feature Selection Methods

11.1.

Selection Rationale

11.1.1.

Curse of Dimensionality

11.1.2.

Overfitting Prevention

11.1.3.

Computational Efficiency

11.1.4.

Model Interpretability

11.1.5.

Noise Reduction

11.2.

11.2.1.

Statistical Tests

11.2.1.1.

Chi-Square Test

11.2.1.2.

11.2.1.3.

Mutual Information

11.2.1.4.

Information Gain

11.2.2.

Correlation-Based Selection

11.2.2.1.

Pearson Correlation

11.2.2.2.

Spearman Correlation

11.2.2.3.

Distance Correlation

11.2.3.

Variance-Based Selection

11.2.3.1.

Variance Threshold

11.2.3.2.

Coefficient of Variation

11.2.4.

Univariate Selection

11.2.4.1.

11.2.4.2.

SelectPercentile

11.2.4.3.

11.2.4.4.

11.2.4.5.

11.3.

Wrapper Methods

11.3.1.

Sequential Selection

11.3.1.1.

Forward Selection

11.3.1.2.

Backward Elimination

11.3.1.3.

Bidirectional Selection

11.3.2.

Recursive Feature Elimination

11.3.3.

Exhaustive Search

11.3.4.

Genetic Algorithms

11.4.

Embedded Methods

11.4.1.

Regularization-Based

11.4.1.1.

L1 Regularization (Lasso)

11.4.1.2.

L2 Regularization (Ridge)

11.4.1.3.

11.4.2.

Tree-Based Importance

11.4.2.1.

Gini Importance

11.4.2.2.

Permutation Importance

11.4.2.3.

Drop-Column Importance

11.4.3.

Model-Specific Selection

11.4.3.1.

Linear Model Coefficients

11.4.3.2.

Neural Network Weights

11.5.

Hybrid Approaches

11.5.1.

Multi-Stage Selection

11.5.2.

Ensemble Selection Methods

11.5.3.

Stability Selection

Previous

10. Geospatial Feature Engineering

Go to top

Next

12. Dimensionality Reduction