Mathematics for Machine Learning and Data Science

Linear Algebra: The Language of Data

2.1.

Vectors

2.1.1.

Definition of Vectors

2.1.1.1.

Geometric Interpretation

2.1.1.2.

Algebraic Interpretation

2.1.1.3.

Column and Row Vector Notation

2.1.2.

Vector Operations

2.1.2.1.

Vector Addition

2.1.2.2.

Vector Subtraction

2.1.2.3.

Scalar Multiplication

2.1.2.4.

Properties of Vector Operations

2.1.3.

Dot Product

2.1.3.1.

Definition and Calculation

2.1.3.2.

Geometric Interpretation

2.1.3.3.

Angle Between Vectors

2.1.3.4.

Orthogonality

2.1.4.

Cross Product

2.1.4.1.

Definition and Calculation

2.1.4.2.

Geometric Interpretation

2.1.4.3.

Properties of Cross Product

2.1.5.

2.1.5.1.

2.1.5.2.

2.1.5.3.

2.1.5.4.

2.1.5.5.

2.1.5.6.

Unit Vectors and Normalization

2.1.6.

Vector Projections

2.1.6.1.

Scalar Projection

2.1.6.2.

Vector Projection

2.1.6.3.

Orthogonal Projections

2.1.6.4.

Gram-Schmidt Process

2.2.

Matrices

2.2.1.

Definition of Matrices

2.2.1.1.

2.2.1.2.

2.2.1.3.

2.2.2.

2.2.2.1.

Matrix Addition and Subtraction

2.2.2.2.

Scalar Multiplication

2.2.2.3.

Matrix Multiplication

2.2.2.4.

Matrix-Vector Multiplication

2.2.2.5.

Properties of Matrix Operations

2.2.3.

Special Types of Matrices

2.2.3.1.

Zero Matrix

2.2.3.2.

Identity Matrix

2.2.3.3.

Diagonal Matrix

2.2.3.4.

Upper and Lower Triangular Matrices

2.2.3.5.

2.2.3.6.

2.2.3.7.

2.2.3.8.

2.2.3.9.

2.2.3.10.

Positive Definite Matrix

2.2.4.

Matrix Transpose

2.2.4.1.

Definition and Properties

2.2.4.2.

Transpose of Products

2.2.5.

Matrix Inverse

2.2.5.1.

Definition of Invertible Matrix

2.2.5.2.

Properties of Matrix Inverse

2.2.5.3.

Methods for Computing Inverse

2.2.5.4.

Pseudoinverse

2.2.6.

Matrix Determinant

2.2.6.1.

Definition for 2x2 and 3x3 Matrices

2.2.6.2.

General Definition

2.2.6.3.

Properties of Determinants

2.2.6.4.

Geometric Interpretation

2.2.7.

Matrix Trace

2.2.7.1.

Definition and Properties

2.3.

Systems of Linear Equations

2.3.1.

Matrix Representation of Linear Systems

2.3.2.

Augmented Matrix

2.3.3.

Gaussian Elimination

2.3.3.1.

Elementary Row Operations

2.3.3.2.

Row Echelon Form

2.3.3.3.

Reduced Row Echelon Form

2.3.4.

Solutions to Linear Systems

2.3.4.1.

Unique Solution

2.3.4.2.

No Solution

2.3.4.3.

Infinitely Many Solutions

2.3.4.4.

Homogeneous Systems

2.3.5.

Matrix Rank and Nullity

2.4.

Vector Spaces

2.4.1.

Definition of Vector Space

2.4.1.1.

Vector Space Axioms

2.4.1.2.

Examples of Vector Spaces

2.4.2.

Subspaces

2.4.2.1.

Definition and Properties

2.4.2.2.

Subspace Test

2.4.2.3.

Examples of Subspaces

2.4.3.

Linear Combinations

2.4.4.

Span of Vectors

2.4.5.

Linear Independence and Dependence

2.4.5.1.

Definition and Tests

2.4.5.2.

Linear Independence Test

2.4.6.

2.4.6.1.

2.4.6.2.

2.4.6.3.

2.4.6.4.

Dimension of Vector Space

2.4.7.

Four Fundamental Subspaces

2.4.7.1.

2.4.7.2.

2.4.7.3.

2.4.7.4.

2.4.8.

2.5.

Linear Transformations

2.5.1.

Definition of Linear Transformation

2.5.2.

Matrix Representation of Linear Transformations

2.5.3.

Kernel and Image

2.5.4.

Composition of Linear Transformations

2.5.5.

Invertible Linear Transformations

2.6.

Eigenvalues and Eigenvectors

2.6.1.

Definition and Geometric Interpretation

2.6.2.

Characteristic Polynomial

2.6.3.

Characteristic Equation

2.6.4.

Computing Eigenvalues and Eigenvectors

2.6.5.

Properties of Eigenvalues and Eigenvectors

2.6.6.

Algebraic and Geometric Multiplicity

2.6.7.

Eigenspaces

2.6.8.

Diagonalization

2.6.8.1.

Conditions for Diagonalizability

2.6.8.2.

Diagonalization Process

2.6.9.

Similar Matrices

2.7.

Matrix Decompositions

2.7.1.

LU Decomposition

2.7.1.1.

Definition and Algorithm

2.7.1.2.

Partial Pivoting

2.7.2.

QR Decomposition

2.7.2.1.

Gram-Schmidt Process

2.7.2.2.

Householder Reflections

2.7.3.

Singular Value Decomposition

2.7.3.1.

Definition and Components

2.7.3.2.

Geometric Interpretation

2.7.3.3.

Computing SVD

2.7.3.4.

Applications in Data Science

2.7.3.5.

Low-Rank Approximation

2.7.4.

Eigendecomposition

2.7.4.1.

Spectral Theorem

2.7.5.

Cholesky Decomposition

2.7.5.1.

Definition and Requirements

2.7.5.2.

Algorithm

2.7.6.

Schur Decomposition

2.8.

Tensors

2.8.1.

Definition and Notation

2.8.2.

Tensor Order and Rank

2.8.3.

Tensor Operations

2.8.3.1.

Addition and Scalar Multiplication

2.8.3.2.

Tensor Product

2.8.3.3.

Tensor Contraction

2.8.3.4.

Tensor Reshaping

2.8.4.

Tensor Decompositions

2.8.4.1.

CP Decomposition

2.8.4.2.

Tucker Decomposition

1. Foundations of Mathematical Notation and Concepts

Go to top

3. Multivariable Calculus: The Tools for Optimization