Natural Language Processing (NLP)

  1. Attention Mechanisms and Transformers
    1. Attention Fundamentals
      1. Motivation and Intuition
        1. Query-Key-Value Framework
          1. Attention Weights and Alignment
          2. Attention Variants
            1. Additive Attention
              1. Multiplicative Attention
                1. Scaled Dot-Product Attention
                  1. Multi-Head Attention
                  2. Self-Attention
                    1. Intra-sequence Dependencies
                      1. Positional Information
                        1. Computational Complexity
                        2. Transformer Architecture
                          1. Encoder-Decoder Structure
                            1. Positional Encodings
                              1. Layer Normalization
                                1. Residual Connections
                                  1. Feed-Forward Networks
                                  2. Training Transformers
                                    1. Teacher Forcing
                                      1. Masked Language Modeling
                                        1. Autoregressive Generation

                                      Previous

                                      9. Recurrent Neural Networks

                                      Go to top

                                      Next

                                      11. Pre-trained Language Models