Computational Linguistics

  1. Statistical and Machine Learning Methods
    1. Probability Theory for Language Modeling
      1. Language Models
        1. N-gram Models
          1. Markov Assumption
            1. Parameter Estimation
            2. Smoothing Techniques
              1. Add-One Smoothing
                1. Good-Turing Discounting
                  1. Kneser-Ney Smoothing
                    1. Modified Kneser-Ney
                    2. Evaluation Metrics
                      1. Perplexity
                        1. Cross-Entropy
                          1. Bits per Character
                        2. Classical Machine Learning
                          1. Supervised Learning
                            1. Classification Algorithms
                              1. Regression Methods
                                1. Feature Engineering
                                  1. Model Selection
                                  2. Unsupervised Learning
                                    1. Clustering Algorithms
                                      1. Dimensionality Reduction
                                        1. Latent Variable Models
                                        2. Semi-Supervised Learning
                                          1. Self-Training
                                            1. Co-Training
                                              1. Graph-Based Methods
                                            2. Sequence Modeling
                                              1. Hidden Markov Models
                                                1. Model Structure
                                                  1. Forward-Backward Algorithm
                                                    1. Viterbi Algorithm
                                                      1. Baum-Welch Training
                                                      2. Conditional Random Fields
                                                        1. Linear-Chain CRFs
                                                          1. Feature Functions
                                                            1. Inference and Training
                                                              1. Semi-CRFs
                                                              2. Maximum Entropy Models
                                                                1. Feature-Based Modeling
                                                                  1. Parameter Estimation
                                                                    1. Regularization
                                                                  2. Neural Network Architectures
                                                                    1. Feedforward Networks
                                                                      1. Multi-Layer Perceptrons
                                                                        1. Activation Functions
                                                                          1. Backpropagation
                                                                          2. Recurrent Neural Networks
                                                                            1. Vanilla RNNs
                                                                              1. Vanishing Gradient Problem
                                                                                1. Bidirectional RNNs
                                                                                2. Long Short-Term Memory
                                                                                  1. Memory Cells and Gates
                                                                                    1. Variants and Extensions
                                                                                      1. Training Considerations
                                                                                      2. Gated Recurrent Units
                                                                                        1. Simplified Architecture
                                                                                          1. Comparison with LSTMs
                                                                                          2. Convolutional Neural Networks
                                                                                            1. Convolution for Text
                                                                                              1. Pooling Operations
                                                                                                1. Text Classification Applications
                                                                                                2. Attention Mechanisms
                                                                                                  1. Attention Functions
                                                                                                    1. Self-Attention
                                                                                                      1. Multi-Head Attention
                                                                                                      2. Transformer Architecture
                                                                                                        1. Encoder-Decoder Structure
                                                                                                          1. Positional Encoding
                                                                                                            1. Layer Normalization
                                                                                                              1. Pre-Training and Fine-Tuning

                                                                                                          Previous

                                                                                                          8. Corpus Linguistics and Data

                                                                                                          Go to top

                                                                                                          Next

                                                                                                          10. Advanced Topics and Applications