Deep Learning for Audio Processing

  1. Advanced Models and Techniques
    1. Generative Models for Audio
      1. Autoregressive Models
        1. WaveNet Architecture
          1. Dilated Causal Convolutions
            1. Conditioning Mechanisms
              1. Global and Local Conditioning
              2. SampleRNN
                1. Hierarchical Structure
                  1. Multi-scale Generation
                  2. WaveRNN
                    1. Efficient Autoregressive Generation
                      1. Sparse Generation
                      2. Parallel WaveNet
                        1. Probability Density Distillation
                          1. Fast Parallel Generation
                        2. Generative Adversarial Networks
                          1. GAN Fundamentals for Audio
                            1. Generator and Discriminator Design
                              1. Adversarial Loss Functions
                              2. WaveGAN
                                1. Raw Audio Generation
                                  1. 1D Convolutional Architecture
                                  2. SpecGAN
                                    1. Spectrogram Generation
                                      1. Post-processing to Audio
                                      2. MelGAN
                                        1. Mel-spectrogram Conditioning
                                          1. Efficient Vocoding
                                          2. HiFi-GAN
                                            1. High-fidelity Audio Generation
                                              1. Multi-scale Discriminators
                                            2. Variational Autoencoders
                                              1. VAE Fundamentals
                                                1. Encoder-Decoder Architecture
                                                  1. Latent Space Modeling
                                                    1. Variational Inference
                                                    2. β-VAE for Audio
                                                      1. Disentangled Representations
                                                        1. Controllable Generation
                                                        2. VQ-VAE for Audio
                                                          1. Vector Quantization
                                                            1. Discrete Latent Representations
                                                            2. Hierarchical VAEs
                                                              1. Multi-level Latent Variables
                                                                1. Structured Generation
                                                              2. Flow-based Models
                                                                1. Normalizing Flows
                                                                  1. Invertible Transformations
                                                                    1. Exact Likelihood Computation
                                                                    2. WaveGlow
                                                                      1. Flow-based Vocoding
                                                                        1. Parallel Generation
                                                                        2. FloWaveNet
                                                                          1. Flow and WaveNet Combination
                                                                            1. High-quality Synthesis
                                                                          2. Diffusion Models
                                                                            1. Denoising Diffusion Probabilistic Models
                                                                              1. Forward and Reverse Processes
                                                                                1. Noise Scheduling
                                                                                2. DiffWave
                                                                                  1. Diffusion for Audio Generation
                                                                                    1. Unconditional and Conditional Generation
                                                                                    2. Grad-TTS
                                                                                      1. Diffusion for Text-to-Speech
                                                                                        1. Score-based Generation
                                                                                    3. Audio Source Separation
                                                                                      1. Problem Formulation
                                                                                        1. Mixing Models
                                                                                          1. Linear Instantaneous Mixing
                                                                                            1. Convolutive Mixing
                                                                                              1. Non-linear Mixing
                                                                                              2. Source Separation Objectives
                                                                                                1. Perfect Reconstruction
                                                                                                  1. Perceptual Quality
                                                                                                    1. Source Isolation
                                                                                                  2. Classical Separation Methods
                                                                                                    1. Independent Component Analysis
                                                                                                      1. Statistical Independence
                                                                                                        1. Non-Gaussian Sources
                                                                                                        2. Non-negative Matrix Factorization
                                                                                                          1. Parts-based Decomposition
                                                                                                            1. Sparsity Constraints
                                                                                                            2. Principal Component Analysis
                                                                                                              1. Dimensionality Reduction
                                                                                                                1. Orthogonal Components
                                                                                                              2. Deep Learning Separation
                                                                                                                1. Supervised Separation
                                                                                                                  1. Mask Estimation
                                                                                                                    1. Direct Signal Estimation
                                                                                                                      1. Permutation Problem
                                                                                                                      2. Deep Clustering
                                                                                                                        1. Embedding-based Separation
                                                                                                                          1. K-means Clustering
                                                                                                                          2. Permutation Invariant Training
                                                                                                                            1. Utterance-level PIT
                                                                                                                              1. Scale-invariant Training
                                                                                                                            2. Music Source Separation
                                                                                                                              1. Vocal Separation
                                                                                                                                1. Lead Vocal Extraction
                                                                                                                                  1. Harmony Separation
                                                                                                                                  2. Instrument Separation
                                                                                                                                    1. Drum Separation
                                                                                                                                      1. Bass Separation
                                                                                                                                        1. Harmonic-Percussive Separation
                                                                                                                                        2. Stem Separation
                                                                                                                                          1. Multi-track Separation
                                                                                                                                            1. Professional Audio Applications
                                                                                                                                          2. Speech Separation
                                                                                                                                            1. Single-channel Speech Separation
                                                                                                                                              1. Monaural Source Separation
                                                                                                                                                1. Deep Learning Approaches
                                                                                                                                                2. Multi-channel Speech Separation
                                                                                                                                                  1. Beamforming Integration
                                                                                                                                                    1. Spatial Information Utilization
                                                                                                                                                    2. Speaker-independent Separation
                                                                                                                                                      1. Universal Separation Models
                                                                                                                                                        1. Adaptation Techniques
                                                                                                                                                    3. Self-Supervised and Unsupervised Learning
                                                                                                                                                      1. Contrastive Learning
                                                                                                                                                        1. Contrastive Predictive Coding
                                                                                                                                                          1. Predictive Coding Framework
                                                                                                                                                            1. Negative Sampling
                                                                                                                                                            2. SimCLR for Audio
                                                                                                                                                              1. Data Augmentation Strategies
                                                                                                                                                                1. Contrastive Loss Functions
                                                                                                                                                                2. Audio Representation Learning
                                                                                                                                                                  1. Temporal Contrastive Learning
                                                                                                                                                                    1. Cross-modal Contrastive Learning
                                                                                                                                                                  2. Masked Language Modeling for Audio
                                                                                                                                                                    1. Masked Acoustic Modeling
                                                                                                                                                                      1. Random Masking Strategies
                                                                                                                                                                        1. Reconstruction Objectives
                                                                                                                                                                        2. wav2vec 2.0
                                                                                                                                                                          1. Quantized Representations
                                                                                                                                                                            1. Contrastive Learning
                                                                                                                                                                            2. HuBERT
                                                                                                                                                                              1. Hidden Unit BERT
                                                                                                                                                                                1. Iterative Refinement
                                                                                                                                                                              2. Pretext Tasks
                                                                                                                                                                                1. Temporal Order Prediction
                                                                                                                                                                                  1. Sequence Order Learning
                                                                                                                                                                                    1. Shuffle Detection
                                                                                                                                                                                    2. Speed Prediction
                                                                                                                                                                                      1. Playback Speed Classification
                                                                                                                                                                                        1. Temporal Dynamics Learning
                                                                                                                                                                                        2. Rotation Prediction
                                                                                                                                                                                          1. Spectrogram Rotation
                                                                                                                                                                                            1. Spatial Relationship Learning
                                                                                                                                                                                          2. Multi-modal Self-supervision
                                                                                                                                                                                            1. Audio-Visual Learning
                                                                                                                                                                                              1. Cross-modal Correspondence
                                                                                                                                                                                                1. Synchronization Learning
                                                                                                                                                                                                2. Audio-Text Learning
                                                                                                                                                                                                  1. Speech-Text Alignment
                                                                                                                                                                                                    1. Semantic Correspondence
                                                                                                                                                                                                3. Transfer Learning and Pre-trained Models
                                                                                                                                                                                                  1. Transfer Learning Strategies
                                                                                                                                                                                                    1. Feature Extraction
                                                                                                                                                                                                      1. Frozen Pre-trained Features
                                                                                                                                                                                                        1. Feature Concatenation
                                                                                                                                                                                                        2. Fine-tuning
                                                                                                                                                                                                          1. Layer-wise Learning Rates
                                                                                                                                                                                                            1. Gradual Unfreezing
                                                                                                                                                                                                            2. Domain Adaptation
                                                                                                                                                                                                              1. Domain Shift Handling
                                                                                                                                                                                                                1. Adversarial Domain Adaptation
                                                                                                                                                                                                              2. Pre-trained Audio Models
                                                                                                                                                                                                                1. VGGish
                                                                                                                                                                                                                  1. YouTube-8M Pre-training
                                                                                                                                                                                                                    1. Audio Event Classification
                                                                                                                                                                                                                    2. PANNs
                                                                                                                                                                                                                      1. AudioSet Pre-training
                                                                                                                                                                                                                        1. Large-scale Audio Recognition
                                                                                                                                                                                                                        2. wav2vec Models
                                                                                                                                                                                                                          1. Self-supervised Pre-training
                                                                                                                                                                                                                            1. Speech Representation Learning
                                                                                                                                                                                                                            2. CLAP
                                                                                                                                                                                                                              1. Contrastive Language-Audio Pre-training
                                                                                                                                                                                                                                1. Cross-modal Representations
                                                                                                                                                                                                                              2. Cross-domain Transfer
                                                                                                                                                                                                                                1. Image-to-Audio Transfer
                                                                                                                                                                                                                                  1. CNN Architecture Transfer
                                                                                                                                                                                                                                    1. Spectrogram as Image
                                                                                                                                                                                                                                    2. Speech-to-Music Transfer
                                                                                                                                                                                                                                      1. Domain Adaptation Techniques
                                                                                                                                                                                                                                        1. Feature Space Alignment
                                                                                                                                                                                                                                        2. Language-to-Audio Transfer
                                                                                                                                                                                                                                          1. Transformer Architecture Transfer
                                                                                                                                                                                                                                            1. Sequence Modeling Transfer
                                                                                                                                                                                                                                          2. Few-shot and Zero-shot Learning
                                                                                                                                                                                                                                            1. Meta-learning for Audio
                                                                                                                                                                                                                                              1. Model-Agnostic Meta-Learning
                                                                                                                                                                                                                                                1. Prototypical Networks
                                                                                                                                                                                                                                                2. Zero-shot Audio Classification
                                                                                                                                                                                                                                                  1. Semantic Embeddings
                                                                                                                                                                                                                                                    1. Attribute-based Classification
                                                                                                                                                                                                                                                    2. Few-shot Adaptation
                                                                                                                                                                                                                                                      1. Rapid Adaptation Techniques
                                                                                                                                                                                                                                                        1. Support Set Utilization

                                                                                                                                                                                                                                                  Previous

                                                                                                                                                                                                                                                  4. Key Applications and Tasks

                                                                                                                                                                                                                                                  Go to top

                                                                                                                                                                                                                                                  Next

                                                                                                                                                                                                                                                  6. Practical Implementation and Evaluation