Deep Learning with PyTorch

  1. Training and Evaluating Models
    1. The Training Loop
      1. Training Loop Structure
        1. Epoch and Batch Iteration
          1. Mode Switching
          2. Forward Pass
            1. Input Processing
              1. Model Prediction
              2. Loss Calculation
                1. Loss Function Application
                  1. Loss Aggregation
                  2. Backward Pass
                    1. Gradient Computation
                      1. Backpropagation
                      2. Parameter Updates
                        1. Optimizer Step
                          1. Gradient Zeroing
                          2. Progress Tracking
                            1. Loss Monitoring
                              1. Metric Calculation
                            2. Optimizers (torch.optim)
                              1. Gradient Descent Variants
                                1. Stochastic Gradient Descent (SGD)
                                  1. Learning Rate
                                    1. Momentum
                                      1. Weight Decay
                                        1. Nesterov Momentum
                                        2. Mini-batch Gradient Descent
                                        3. Adaptive Optimizers
                                          1. Adam
                                            1. Adaptive Learning Rates
                                              1. Beta Parameters
                                                1. Epsilon Parameter
                                                2. AdamW
                                                  1. Decoupled Weight Decay
                                                  2. RMSprop
                                                    1. Moving Average of Squared Gradients
                                                    2. Adagrad
                                                      1. Accumulated Squared Gradients
                                                      2. Adadelta
                                                        1. Adaptive Learning Rate
                                                      3. Learning Rate Scheduling
                                                        1. StepLR
                                                          1. MultiStepLR
                                                            1. ExponentialLR
                                                              1. CosineAnnealingLR
                                                                1. ReduceLROnPlateau
                                                                  1. Custom Schedulers
                                                                  2. Optimizer State Management
                                                                    1. State Dictionary
                                                                      1. Checkpoint Saving and Loading
                                                                    2. Model Evaluation
                                                                      1. Evaluation Mode
                                                                        1. model.eval() vs model.train()
                                                                          1. Batch Normalization Behavior
                                                                            1. Dropout Behavior
                                                                            2. Inference Process
                                                                              1. torch.no_grad() Context
                                                                                1. Prediction Generation
                                                                                  1. Batch Processing
                                                                                  2. Evaluation Metrics
                                                                                    1. Classification Metrics
                                                                                      1. Accuracy
                                                                                        1. Precision
                                                                                          1. Recall
                                                                                            1. F1-Score
                                                                                              1. Confusion Matrix
                                                                                                1. ROC Curve and AUC
                                                                                                  1. Precision-Recall Curve
                                                                                                  2. Regression Metrics
                                                                                                    1. Mean Squared Error
                                                                                                      1. Mean Absolute Error
                                                                                                        1. R-squared
                                                                                                          1. Mean Absolute Percentage Error
                                                                                                        2. Cross-Validation
                                                                                                          1. K-Fold Cross-Validation
                                                                                                            1. Stratified Cross-Validation
                                                                                                              1. Time Series Cross-Validation
                                                                                                            2. Overfitting and Underfitting
                                                                                                              1. Identifying Overfitting
                                                                                                                1. Training vs Validation Performance
                                                                                                                  1. Learning Curves
                                                                                                                  2. Identifying Underfitting
                                                                                                                    1. Poor Training Performance
                                                                                                                      1. High Bias Indicators
                                                                                                                      2. Bias-Variance Tradeoff
                                                                                                                        1. Model Complexity
                                                                                                                          1. Generalization Gap
                                                                                                                        2. Regularization Techniques
                                                                                                                          1. Weight Regularization
                                                                                                                            1. L1 Regularization (Lasso)
                                                                                                                              1. L2 Regularization (Ridge)
                                                                                                                                1. Elastic Net
                                                                                                                                2. Structural Regularization
                                                                                                                                  1. Dropout
                                                                                                                                    1. Standard Dropout
                                                                                                                                      1. Spatial Dropout
                                                                                                                                        1. Dropout Scheduling
                                                                                                                                        2. Batch Normalization
                                                                                                                                          1. Internal Covariate Shift
                                                                                                                                            1. Regularization Effect
                                                                                                                                          2. Early Stopping
                                                                                                                                            1. Validation Loss Monitoring
                                                                                                                                              1. Patience Parameter
                                                                                                                                                1. Best Model Restoration
                                                                                                                                                2. Data Augmentation as Regularization
                                                                                                                                                  1. Ensemble Methods
                                                                                                                                                    1. Model Averaging
                                                                                                                                                      1. Voting Classifiers

                                                                                                                                                  Previous

                                                                                                                                                  4. Data Loading and Processing

                                                                                                                                                  Go to top

                                                                                                                                                  Next

                                                                                                                                                  6. Convolutional Neural Networks (CNNs) for Computer Vision