Information Theory

4.

Channel Capacity and Noisy Communication

4.1.

4.1.1.

Discrete Memoryless Channels

4.1.1.1.

Input and Output Alphabets

4.1.1.2.

Transition Probabilities

4.1.1.3.

Channel Matrix Representation

4.1.2.

Specific Channel Types

4.1.2.1.

Binary Symmetric Channel

4.1.2.2.

Binary Erasure Channel

4.1.2.3.

4.1.2.4.

q-ary Symmetric Channel

4.1.3.

Channel Characteristics

4.1.3.1.

4.1.3.2.

4.1.3.3.

Time-Varying Channels

4.2.

Channel Capacity

4.2.1.

Definition as Maximum Mutual Information

4.2.2.

Capacity-Achieving Input Distributions

4.2.3.

Computing Capacity

4.2.3.1.

Analytical Methods

4.2.3.2.

Numerical Optimization

4.2.3.3.

Blahut-Arimoto Algorithm

4.2.4.

Capacity of Specific Channels

4.2.4.1.

BSC Capacity Calculation

4.2.4.2.

BEC Capacity Calculation

4.2.4.3.

Symmetric Channel Capacity

4.3.

Noisy-Channel Coding Theorem

4.3.1.

Statement and Interpretation

4.3.2.

Achievability Proof

4.3.2.1.

Random Coding Argument

4.3.2.2.

Typical Set Decoding

4.3.2.3.

Error Probability Analysis

4.3.3.

4.3.3.1.

Fano's Inequality

4.3.3.2.

Information-Theoretic Bounds

4.3.4.

Implications for Communication System Design

4.4.

Continuous Channels

4.4.1.

Additive White Gaussian Noise Channel

4.4.1.1.

4.4.1.2.

Power Constraints

4.4.1.3.

Bandwidth Limitations

4.4.2.

Shannon-Hartley Theorem

4.4.2.1.

Capacity Formula

4.4.2.2.

Signal-to-Noise Ratio Trade-offs

4.4.2.3.

Bandwidth-Power Trade-offs

4.4.3.

Water-Filling Principle

4.4.3.1.

Parallel Gaussian Channels

4.4.3.2.

Optimal Power Allocation

4.4.3.3.

Frequency-Selective Channels

Previous

3. Source Coding and Lossless Compression

Go to top

Next

5. Error Control Coding