Information Theory

  1. Rate-Distortion Theory
    1. Lossy Compression Fundamentals
      1. Motivation for Lossy Compression
        1. Limitations of Lossless Methods
          1. Perceptual Considerations
            1. Application Requirements
            2. Distortion Measures
              1. Hamming Distortion
                1. Squared Error Distortion
                  1. Absolute Error Distortion
                    1. Perceptual Distortion Measures
                  2. Rate-Distortion Function
                    1. Definition and Properties
                      1. Convexity
                        1. Monotonicity
                          1. Continuity
                          2. Computing Rate-Distortion Functions
                            1. Blahut-Arimoto Algorithm
                              1. Analytical Solutions
                                1. Parametric Representations
                                2. Rate-Distortion for Specific Sources
                                  1. Bernoulli Source
                                    1. Gaussian Source
                                      1. Uniform Source
                                    2. Rate-Distortion Theorem
                                      1. Achievability Proof
                                        1. Random Coding Argument
                                          1. Joint Typicality
                                            1. Covering Lemma
                                            2. Converse Proof
                                              1. Information-Theoretic Bounds
                                                1. Operational Significance
                                              2. Quantization Theory
                                                1. Scalar Quantization
                                                  1. Uniform Quantizers
                                                    1. Non-uniform Quantizers
                                                      1. Lloyd-Max Quantizers
                                                        1. Companding
                                                        2. Vector Quantization
                                                          1. Codebook Design
                                                            1. Nearest Neighbor Encoding
                                                              1. Centroid Conditions
                                                                1. LBG Algorithm
                                                                2. Entropy-Constrained Quantization
                                                                  1. Optimal Quantizer Design
                                                                    1. High-Rate Approximations
                                                                      1. Operational Rate-Distortion Performance

                                                                  Previous

                                                                  5. Error Control Coding

                                                                  Go to top

                                                                  Next

                                                                  7. Information Theory in Continuous Settings