Deep Learning for Audio Processing

2.

Audio Data Representation and Preprocessing

2.1.

Raw Audio Waveform Processing

2.1.1.

Digital Audio Structure

2.1.1.1.

Sample Arrays and Data Types

2.1.1.2.

Mono vs Stereo vs Multi-channel

2.1.1.3.

Memory Layout and Storage

2.1.2.

Temporal Segmentation

2.1.2.1.

Fixed-length Windowing

2.1.2.2.

Overlapping Windows

2.1.2.3.

Hop Length Selection

2.1.2.4.

Boundary Handling

2.1.3.

Window Functions

2.1.3.1.

Rectangular Window

2.1.3.2.

2.1.3.3.

2.1.3.4.

Blackman Window

2.1.3.5.

2.1.3.6.

Window Selection Criteria

2.2.

Time-Frequency Representations

2.2.1.

Short-Time Fourier Transform

2.2.1.1.

STFT Computation

2.2.1.2.

Window Size vs Time Resolution

2.2.1.3.

Frequency Resolution Considerations

2.2.1.4.

Overlap-Add Reconstruction

2.2.2.

Spectrogram Variants

2.2.2.1.

Linear Frequency Spectrogram

2.2.2.1.1.

Construction from STFT

2.2.2.1.2.

Magnitude and Phase Information

2.2.2.2.

Logarithmic Frequency Spectrogram

2.2.2.2.1.

Log-scale Benefits

2.2.2.2.2.

Perceptual Relevance

2.2.2.3.

Mel-scale Spectrogram

2.2.2.3.1.

Mel Filter Bank Design

2.2.2.3.2.

Perceptual Frequency Scaling

2.2.2.3.3.

Implementation Details

2.2.2.4.

Bark-scale Spectrogram

2.2.2.4.1.

Critical Band Theory

2.2.2.4.2.

Bark Scale Definition

2.2.3.

Advanced Time-Frequency Methods

2.2.3.1.

Constant-Q Transform

2.2.3.1.1.

Variable Time-Frequency Resolution

2.2.3.1.2.

Musical Applications

2.2.3.1.3.

Implementation Considerations

2.2.3.2.

Gammatone Filterbank

2.2.3.2.1.

Auditory Filter Modeling

2.2.3.2.2.

Cochlear Frequency Analysis

2.2.3.3.

2.2.3.3.1.

Pitch Class Representation

2.2.3.3.2.

Octave Equivalence

2.2.3.3.3.

Harmonic Analysis

2.3.

Feature Extraction and Engineering

2.3.1.

Low-level Audio Features

2.3.1.1.

Temporal Features

2.3.1.1.1.

Root Mean Square Energy

2.3.1.1.2.

Zero-Crossing Rate

2.3.1.1.3.

Spectral Centroid

2.3.1.1.4.

Spectral Bandwidth

2.3.1.2.

Spectral Features

2.3.1.2.1.

Spectral Contrast

2.3.1.2.2.

Spectral Flatness

2.3.1.2.3.

Spectral Rolloff

2.3.1.2.4.

2.3.1.3.

Cepstral Features

2.3.1.3.1.

MFCC Computation Pipeline

2.3.1.3.2.

Delta and Delta-Delta Coefficients

2.3.1.3.3.

Liftering and Post-processing

2.3.2.

Mid-level Features

2.3.2.1.

Chroma Features

2.3.2.1.1.

Pitch Class Profiles

2.3.2.1.2.

Harmonic Content Analysis

2.3.2.1.3.

Key and Chord Recognition

2.3.2.2.

Tonnetz Features

2.3.2.2.1.

Harmonic Network Representation

2.3.2.2.2.

Tonal Centroid Features

2.3.2.3.

Rhythm Features

2.3.2.3.1.

Tempo Estimation

2.3.2.3.2.

2.3.2.3.3.

Onset Detection

2.3.3.

High-level Features

2.3.3.1.

Structural Features

2.3.3.1.1.

Segment Boundaries

2.3.3.1.2.

Repetition Analysis

2.3.3.1.3.

2.3.3.2.

Semantic Features

2.3.3.2.1.

Mood and Emotion

2.3.3.2.2.

Genre Characteristics

2.3.3.2.3.

Instrument Presence

2.4.

Data Preprocessing Techniques

2.4.1.

Amplitude Normalization

2.4.1.1.

Peak Normalization

2.4.1.2.

RMS Normalization

2.4.1.3.

Loudness Normalization

2.4.1.4.

Dynamic Range Compression

2.4.2.

Statistical Normalization

2.4.2.1.

Z-score Standardization

2.4.2.2.

Min-max Scaling

2.4.2.3.

2.4.2.4.

Per-channel Normalization

2.4.3.

Silence and Noise Handling

2.4.3.1.

Voice Activity Detection

2.4.3.2.

Silence Removal Algorithms

2.4.3.3.

Noise Floor Estimation

2.4.3.4.

Signal-to-Noise Ratio Enhancement

2.4.4.

Temporal Alignment

2.4.4.1.

Audio Synchronization

2.4.4.2.

Time Stretching

2.4.4.3.

2.4.4.4.

Cross-correlation Alignment

2.5.

Data Augmentation Strategies

2.5.1.

Time-Domain Augmentation

2.5.1.1.

2.5.1.2.

Time Stretching

2.5.1.3.

Speed Perturbation

2.5.1.4.

Polarity Inversion

2.5.2.

Frequency-Domain Augmentation

2.5.2.1.

2.5.2.2.

Formant Shifting

2.5.2.3.

Frequency Masking

2.5.2.4.

Spectral Subtraction

2.5.3.

Additive Augmentation

2.5.3.1.

Background Noise Addition

2.5.3.2.

Reverberation Simulation

2.5.3.3.

Echo and Delay Effects

2.5.3.4.

Multi-speaker Mixing

2.5.4.

SpecAugment Techniques

2.5.4.1.

2.5.4.2.

Frequency Masking

2.5.4.3.

2.5.4.4.

Adaptive Masking Strategies

2.5.5.

Advanced Augmentation

2.5.5.1.

Room Impulse Response Convolution

2.5.5.2.

Codec Simulation

2.5.5.3.

Channel Effects

2.5.5.4.

Mixup and CutMix for Audio

Previous

1. Foundations of Audio and Deep Learning

Go to top

Next

3. Core Deep Learning Architectures for Audio