Deep Learning with PyTorch

  1. Building Neural Networks with torch.nn
    1. The nn.Module Class
      1. Understanding nn.Module
        1. Base Class for All Neural Networks
          1. Parameter Management
            1. State Management
            2. Creating Custom Modules
              1. Subclassing nn.Module
                1. init() Method Implementation
                  1. forward() Method Implementation
                  2. Module Hierarchy
                    1. Parent and Child Modules
                      1. Module Registration
                        1. Named Modules and Parameters
                        2. Parameter and Buffer Management
                          1. Registering Parameters
                            1. Registering Buffers
                              1. Parameter Initialization
                            2. Linear Layers
                              1. Fully Connected Layers (nn.Linear)
                                1. Input and Output Features
                                  1. Weight Matrix
                                    1. Bias Vector
                                      1. Mathematical Formulation
                                      2. Linear Layer Variations
                                        1. Bilinear Layers
                                          1. Identity Layers
                                        2. Convolutional Layers
                                          1. 1D Convolution (nn.Conv1d)
                                            1. Input and Output Channels
                                              1. Kernel Size
                                                1. Stride and Padding
                                                  1. Dilation
                                                  2. 2D Convolution (nn.Conv2d)
                                                    1. Feature Map Processing
                                                      1. Spatial Dimensions
                                                        1. Parameter Sharing
                                                        2. 3D Convolution (nn.Conv3d)
                                                          1. Transposed Convolution
                                                            1. Upsampling Operations
                                                              1. Deconvolution Concept
                                                              2. Depthwise and Separable Convolutions
                                                              3. Pooling Layers
                                                                1. Max Pooling
                                                                  1. nn.MaxPool1d
                                                                    1. nn.MaxPool2d
                                                                      1. nn.MaxPool3d
                                                                      2. Average Pooling
                                                                        1. nn.AvgPool1d
                                                                          1. nn.AvgPool2d
                                                                            1. nn.AvgPool3d
                                                                            2. Adaptive Pooling
                                                                              1. AdaptiveMaxPool2d
                                                                                1. AdaptiveAvgPool2d
                                                                                2. Global Pooling
                                                                                3. Recurrent Layers
                                                                                  1. Simple RNN (nn.RNN)
                                                                                    1. Vanilla RNN Architecture
                                                                                      1. Hidden State Processing
                                                                                      2. Long Short-Term Memory (nn.LSTM)
                                                                                        1. LSTM Cell Architecture
                                                                                          1. Bidirectional LSTM
                                                                                          2. Gated Recurrent Unit (nn.GRU)
                                                                                            1. GRU Cell Architecture
                                                                                              1. Bidirectional GRU
                                                                                              2. RNN Input and Output Handling
                                                                                                1. Sequence Length Handling
                                                                                                  1. Batch First Option
                                                                                                2. Activation Functions
                                                                                                  1. Linear Activations
                                                                                                    1. Identity Function
                                                                                                    2. Non-linear Activations
                                                                                                      1. Sigmoid
                                                                                                        1. Tanh
                                                                                                          1. ReLU
                                                                                                            1. LeakyReLU
                                                                                                              1. PReLU
                                                                                                                1. ELU
                                                                                                                  1. SELU
                                                                                                                    1. Swish
                                                                                                                      1. GELU
                                                                                                                      2. Softmax and LogSoftmax
                                                                                                                        1. Choosing Activation Functions
                                                                                                                        2. Normalization Layers
                                                                                                                          1. Batch Normalization
                                                                                                                            1. nn.BatchNorm1d
                                                                                                                              1. nn.BatchNorm2d
                                                                                                                                1. nn.BatchNorm3d
                                                                                                                                2. Layer Normalization
                                                                                                                                  1. Instance Normalization
                                                                                                                                    1. Group Normalization
                                                                                                                                    2. Regularization Layers
                                                                                                                                      1. Dropout
                                                                                                                                        1. nn.Dropout
                                                                                                                                          1. nn.Dropout2d
                                                                                                                                            1. nn.Dropout3d
                                                                                                                                            2. AlphaDropout
                                                                                                                                            3. Loss Functions
                                                                                                                                              1. Regression Losses
                                                                                                                                                1. Mean Squared Error (MSELoss)
                                                                                                                                                  1. Mean Absolute Error (L1Loss)
                                                                                                                                                    1. Smooth L1 Loss
                                                                                                                                                      1. Huber Loss
                                                                                                                                                      2. Classification Losses
                                                                                                                                                        1. Cross Entropy Loss
                                                                                                                                                          1. Negative Log Likelihood Loss
                                                                                                                                                            1. Binary Cross Entropy Loss
                                                                                                                                                              1. Focal Loss
                                                                                                                                                              2. Custom Loss Functions
                                                                                                                                                                1. Implementing Custom Losses
                                                                                                                                                                  1. Combining Multiple Losses

                                                                                                                                                              Previous

                                                                                                                                                              2. PyTorch Fundamentals

                                                                                                                                                              Go to top

                                                                                                                                                              Next

                                                                                                                                                              4. Data Loading and Processing