Graph Neural Networks

  1. Core Concepts of Graph Neural Networks
    1. Motivation and Foundations
      1. Limitations of Traditional Neural Networks
        1. Fixed Input Size Requirements
          1. Lack of Permutation Invariance
            1. Inability to Handle Graph Structure
              1. Missing Relational Information
              2. Inductive Biases for Graphs
                1. Permutation Invariance
                  1. Locality Principle
                    1. Shared Parameters
                    2. From Traditional ML to GNNs
                      1. End-to-end Learning
                        1. Representation Learning
                          1. Parameter Sharing
                        2. Message Passing Framework
                          1. Core Message Passing Paradigm
                            1. Neighborhood Aggregation Principle
                              1. Iterative Refinement
                                1. Local to Global Information Flow
                                2. Message Function
                                  1. Definition and Purpose
                                    1. Edge Feature Integration
                                      1. Parameterization Strategies
                                      2. Aggregation Function
                                        1. Permutation Invariance Requirement
                                          1. Common Aggregation Operations
                                            1. Sum Aggregation
                                              1. Mean Aggregation
                                                1. Max Aggregation
                                                  1. Attention-based Aggregation
                                                  2. Set Function Properties
                                                  3. Update Function
                                                    1. Combining Messages with Node Features
                                                      1. Non-linear Transformations
                                                        1. Gating Mechanisms
                                                        2. Readout Function
                                                          1. Graph-level Representations
                                                            1. Pooling Operations
                                                              1. Global Information Extraction
                                                            2. Node Representations and Embeddings
                                                              1. Node Embedding Concept
                                                                1. From Features to Representations
                                                                  1. Learned vs Handcrafted Features
                                                                    1. Embedding Dimensionality
                                                                    2. Multi-layer Representations
                                                                      1. Layer-wise Feature Evolution
                                                                        1. Receptive Field Growth
                                                                          1. Depth vs Width Trade-offs
                                                                          2. Initialization Strategies
                                                                            1. Random Initialization
                                                                              1. Pre-trained Embeddings
                                                                                1. Feature-based Initialization
                                                                              2. Key Design Principles
                                                                                1. Permutation Equivariance
                                                                                  1. Mathematical Definition
                                                                                    1. Implementation Considerations
                                                                                    2. Translation Invariance
                                                                                      1. Scalability Considerations
                                                                                        1. Linear Complexity Goals
                                                                                          1. Memory Efficiency
                                                                                          2. Expressiveness vs Efficiency
                                                                                            1. Theoretical Limitations
                                                                                              1. Practical Trade-offs

                                                                                          Previous

                                                                                          1. Foundations for Understanding GNNs

                                                                                          Go to top

                                                                                          Next

                                                                                          3. Foundational GNN Architectures