Natural Language Processing (NLP)

  1. Pre-trained Language Models
    1. Transfer Learning Paradigm
      1. Pre-training Objectives
        1. Fine-tuning Strategies
          1. Domain Adaptation
          2. Encoder-Only Models
            1. BERT
              1. Masked Language Modeling
                1. Next Sentence Prediction
                  1. Bidirectional Context
                  2. RoBERTa
                    1. Training Optimizations
                      1. Dynamic Masking
                      2. ALBERT
                        1. Parameter Sharing
                          1. Factorized Embeddings
                          2. DeBERTa
                            1. Disentangled Attention
                              1. Enhanced Mask Decoder
                            2. Decoder-Only Models
                              1. GPT Family
                                1. Autoregressive Training
                                  1. Scaling Laws
                                    1. In-Context Learning
                                    2. LLaMA
                                      1. Architectural Improvements
                                        1. Efficient Training
                                        2. PaLM
                                          1. Pathways Architecture
                                            1. Emergent Abilities
                                          2. Encoder-Decoder Models
                                            1. T5
                                              1. Text-to-Text Framework
                                                1. Unified Pre-training
                                                2. BART
                                                  1. Denoising Autoencoder
                                                    1. Text Infilling
                                                    2. UL2
                                                      1. Unified Language Learner
                                                        1. Mixed Denoising
                                                      2. Model Adaptation Techniques
                                                        1. Full Fine-tuning
                                                          1. Parameter-Efficient Methods
                                                            1. LoRA
                                                              1. Adapters
                                                                1. Prompt Tuning
                                                                  1. Prefix Tuning
                                                                  2. In-Context Learning
                                                                    1. Few-Shot Learning

                                                                  Previous

                                                                  10. Attention Mechanisms and Transformers

                                                                  Go to top

                                                                  Next

                                                                  12. Core NLP Applications