Information Theory

  1. Advanced Topics and Applications
    1. Algorithmic Information Theory
      1. Kolmogorov Complexity
        1. Definition and Basic Properties
          1. Incompressibility
            1. Random Strings
            2. Algorithmic Mutual Information
              1. Definition and Properties
                1. Relationship to Shannon Information
                2. Applications
                  1. Data Compression
                    1. Machine Learning Theory
                      1. Computational Complexity
                    2. Information Theory and Statistics
                      1. Fisher Information
                        1. Definition and Properties
                          1. Cramér-Rao Bound
                            1. Efficient Estimators
                            2. Minimum Description Length
                              1. Two-Part Codes
                                1. Universal Models
                                  1. Model Selection
                                  2. Maximum Entropy Methods
                                    1. Principle of Maximum Entropy
                                      1. Exponential Families
                                        1. Applications in Statistical Modeling
                                      2. Information Theory in Machine Learning
                                        1. Information-Theoretic Learning
                                          1. Mutual Information Estimation
                                            1. Feature Selection
                                              1. Clustering
                                              2. Deep Learning Connections
                                                1. Information Bottleneck Principle
                                                  1. Variational Information Maximization
                                                    1. Representation Learning
                                                    2. Generalization Theory
                                                      1. PAC-Bayes Bounds
                                                        1. Information-Theoretic Generalization
                                                          1. Compression-Based Bounds
                                                        2. Quantum Information Theory
                                                          1. Quantum Entropy
                                                            1. Von Neumann Entropy
                                                              1. Quantum Relative Entropy
                                                                1. Quantum Mutual Information
                                                                2. Quantum Channels
                                                                  1. Quantum Channel Capacity
                                                                    1. Entanglement-Assisted Capacity
                                                                      1. Classical-Quantum Channels
                                                                      2. Quantum Error Correction
                                                                        1. Quantum Error Correction Codes
                                                                          1. Stabilizer Codes
                                                                            1. Threshold Theorems

                                                                        Previous

                                                                        8. Network Information Theory

                                                                        Go to top

                                                                        Back to Start

                                                                        1. Foundational Concepts of Information