Graph Neural Networks

  1. GNN Training and Optimization
    1. Training Paradigms
      1. Supervised Learning
        1. Labeled Data Requirements
          1. Task-specific Objectives
          2. Semi-supervised Learning
            1. Limited Label Scenarios
              1. Label Propagation
              2. Unsupervised Learning
                1. Reconstruction Objectives
                  1. Contrastive Learning
                  2. Self-supervised Learning
                    1. Pretext Tasks
                      1. Downstream Fine-tuning
                    2. Loss Functions
                      1. Node-level Losses
                        1. Cross-entropy Loss
                          1. Focal Loss
                            1. Margin-based Loss
                            2. Edge-level Losses
                              1. Ranking Loss
                                1. Contrastive Loss
                                2. Graph-level Losses
                                  1. Classification Loss
                                    1. Regression Loss
                                      1. Reconstruction Loss
                                      2. Regularization Terms
                                        1. L1 and L2 Regularization
                                          1. Graph Regularization
                                            1. Spectral Regularization
                                          2. Optimization Challenges
                                            1. Gradient Flow in Deep GNNs
                                              1. Oversmoothing Problem
                                                1. Causes and Analysis
                                                  1. Mitigation Strategies
                                                    1. Residual Connections
                                                      1. Dense Connections
                                                        1. Jumping Knowledge Networks
                                                      2. Oversquashing Problem
                                                        1. Information Bottlenecks
                                                          1. Long-range Dependencies
                                                          2. Vanishing Gradient Problem
                                                            1. Deep Network Training
                                                              1. Gradient Clipping
                                                            2. Scalability Solutions
                                                              1. Sampling-based Training
                                                                1. Node Sampling
                                                                  1. Uniform Sampling
                                                                    1. Importance Sampling
                                                                    2. Layer-wise Sampling
                                                                      1. Neighbor Sampling
                                                                        1. Control Variate Methods
                                                                        2. Subgraph Sampling
                                                                          1. GraphSAINT
                                                                            1. Cluster-GCN
                                                                          2. Distributed Training
                                                                            1. Data Parallelism
                                                                              1. Model Parallelism
                                                                                1. Communication Optimization
                                                                                2. Memory Optimization
                                                                                  1. Gradient Checkpointing
                                                                                    1. Mixed Precision Training
                                                                                      1. Model Compression

                                                                                  Previous

                                                                                  4. Advanced GNN Architectures

                                                                                  Go to top

                                                                                  Next

                                                                                  6. GNN Applications and Tasks