Information Theory

  1. Channel Capacity and Noisy Communication
    1. Channel Models
      1. Discrete Memoryless Channels
        1. Input and Output Alphabets
          1. Transition Probabilities
            1. Channel Matrix Representation
            2. Specific Channel Types
              1. Binary Symmetric Channel
                1. Binary Erasure Channel
                  1. Z-Channel
                    1. q-ary Symmetric Channel
                    2. Channel Characteristics
                      1. Noise Models
                        1. Channel Memory
                          1. Time-Varying Channels
                        2. Channel Capacity
                          1. Definition as Maximum Mutual Information
                            1. Capacity-Achieving Input Distributions
                              1. Computing Capacity
                                1. Analytical Methods
                                  1. Numerical Optimization
                                    1. Blahut-Arimoto Algorithm
                                    2. Capacity of Specific Channels
                                      1. BSC Capacity Calculation
                                        1. BEC Capacity Calculation
                                          1. Symmetric Channel Capacity
                                        2. Noisy-Channel Coding Theorem
                                          1. Statement and Interpretation
                                            1. Achievability Proof
                                              1. Random Coding Argument
                                                1. Typical Set Decoding
                                                  1. Error Probability Analysis
                                                  2. Converse Proof
                                                    1. Fano's Inequality
                                                      1. Information-Theoretic Bounds
                                                      2. Implications for Communication System Design
                                                      3. Continuous Channels
                                                        1. Additive White Gaussian Noise Channel
                                                          1. Channel Model
                                                            1. Power Constraints
                                                              1. Bandwidth Limitations
                                                              2. Shannon-Hartley Theorem
                                                                1. Capacity Formula
                                                                  1. Signal-to-Noise Ratio Trade-offs
                                                                    1. Bandwidth-Power Trade-offs
                                                                    2. Water-Filling Principle
                                                                      1. Parallel Gaussian Channels
                                                                        1. Optimal Power Allocation
                                                                          1. Frequency-Selective Channels

                                                                      Previous

                                                                      3. Source Coding and Lossless Compression

                                                                      Go to top

                                                                      Next

                                                                      5. Error Control Coding