Computational Linguistics

  1. Mathematical and Computational Foundations
    1. Formal Language Theory
      1. Alphabets, Strings, and Languages
        1. Basic Definitions
          1. Operations on Strings
            1. Language Operations
            2. The Chomsky Hierarchy
              1. Regular Languages
                1. Context-Free Languages
                  1. Context-Sensitive Languages
                    1. Unrestricted Languages
                      1. Computational Complexity Relationships
                      2. Automata Theory
                        1. Finite-State Automata
                          1. Pushdown Automata
                            1. Turing Machines
                              1. Equivalence with Grammar Types
                            2. Probability Theory for Language
                              1. Basic Probability Concepts
                                1. Sample Spaces and Events
                                  1. Conditional Probability
                                    1. Bayes' Theorem
                                    2. Random Variables and Distributions
                                      1. Discrete and Continuous Distributions
                                        1. Expected Value and Variance
                                          1. Common Distributions in NLP
                                          2. Markov Models
                                            1. Markov Assumption
                                              1. Transition Probabilities
                                                1. Stationary Distributions
                                              2. Information Theory
                                                1. Entropy and Information Content
                                                  1. Shannon Entropy
                                                    1. Cross-Entropy
                                                      1. Relative Entropy
                                                      2. Mutual Information
                                                        1. Definition and Properties
                                                          1. Applications in Language Analysis
                                                          2. Coding Theory
                                                            1. Optimal Codes
                                                              1. Huffman Coding
                                                            2. Linear Algebra for Language Processing
                                                              1. Vectors and Vector Spaces
                                                                1. Vector Operations
                                                                  1. Inner Products and Norms
                                                                    1. Orthogonality
                                                                    2. Matrices and Matrix Operations
                                                                      1. Matrix Multiplication
                                                                        1. Eigenvalues and Eigenvectors
                                                                          1. Singular Value Decomposition
                                                                          2. Applications to Language Modeling
                                                                            1. Term-Document Matrices
                                                                              1. Dimensionality Reduction
                                                                                1. Word Embeddings

                                                                            Previous

                                                                            1. Introduction to Computational Linguistics

                                                                            Go to top

                                                                            Next

                                                                            3. Foundational Concepts in Linguistics