Deep Learning with PyTorch

3.

Building Neural Networks with torch.nn

3.1.

The nn.Module Class

3.1.1.

Understanding nn.Module

3.1.1.1.

Base Class for All Neural Networks

3.1.1.2.

Parameter Management

3.1.1.3.

State Management

3.1.2.

Creating Custom Modules

3.1.2.1.

Subclassing nn.Module

3.1.2.2.

init() Method Implementation

3.1.2.3.

forward() Method Implementation

3.1.3.

Module Hierarchy

3.1.3.1.

Parent and Child Modules

3.1.3.2.

Module Registration

3.1.3.3.

Named Modules and Parameters

3.1.4.

Parameter and Buffer Management

3.1.4.1.

Registering Parameters

3.1.4.2.

Registering Buffers

3.1.4.3.

Parameter Initialization

3.2.

3.2.1.

Fully Connected Layers (nn.Linear)

3.2.1.1.

Input and Output Features

3.2.1.2.

3.2.1.3.

3.2.1.4.

Mathematical Formulation

3.2.2.

Linear Layer Variations

3.2.2.1.

Bilinear Layers

3.2.2.2.

Identity Layers

3.3.

Convolutional Layers

3.3.1.

1D Convolution (nn.Conv1d)

3.3.1.1.

Input and Output Channels

3.3.1.2.

3.3.1.3.

Stride and Padding

3.3.1.4.

3.3.2.

2D Convolution (nn.Conv2d)

3.3.2.1.

Feature Map Processing

3.3.2.2.

Spatial Dimensions

3.3.2.3.

Parameter Sharing

3.3.3.

3D Convolution (nn.Conv3d)

3.3.4.

Transposed Convolution

3.3.4.1.

Upsampling Operations

3.3.4.2.

Deconvolution Concept

3.3.5.

Depthwise and Separable Convolutions

3.4.

3.4.1.

3.4.1.1.

3.4.1.2.

3.4.1.3.

3.4.2.

Average Pooling

3.4.2.1.

3.4.2.2.

3.4.2.3.

3.4.3.

Adaptive Pooling

3.4.3.1.

AdaptiveMaxPool2d

3.4.3.2.

AdaptiveAvgPool2d

3.4.4.

3.5.

Recurrent Layers

3.5.1.

Simple RNN (nn.RNN)

3.5.1.1.

Vanilla RNN Architecture

3.5.1.2.

Hidden State Processing

3.5.2.

Long Short-Term Memory (nn.LSTM)

3.5.2.1.

LSTM Cell Architecture

3.5.2.2.

Bidirectional LSTM

3.5.3.

Gated Recurrent Unit (nn.GRU)

3.5.3.1.

GRU Cell Architecture

3.5.3.2.

Bidirectional GRU

3.5.4.

RNN Input and Output Handling

3.5.4.1.

Sequence Length Handling

3.5.4.2.

Batch First Option

3.6.

Activation Functions

3.6.1.

Linear Activations

3.6.1.1.

Identity Function

3.6.2.

Non-linear Activations

3.6.2.1.

3.6.2.2.

3.6.2.3.

3.6.2.4.

3.6.2.5.

3.6.2.6.

3.6.2.7.

3.6.2.8.

3.6.2.9.

3.6.3.

Softmax and LogSoftmax

3.6.4.

Choosing Activation Functions

3.7.

Normalization Layers

3.7.1.

Batch Normalization

3.7.1.1.

3.7.1.2.

3.7.1.3.

3.7.2.

Layer Normalization

3.7.3.

Instance Normalization

3.7.4.

Group Normalization

3.8.

Regularization Layers

3.8.1.

3.8.1.1.

3.8.1.2.

3.8.1.3.

3.8.2.

3.9.

3.9.1.

Regression Losses

3.9.1.1.

Mean Squared Error (MSELoss)

3.9.1.2.

Mean Absolute Error (L1Loss)

3.9.1.3.

3.9.1.4.

3.9.2.

Classification Losses

3.9.2.1.

Cross Entropy Loss

3.9.2.2.

Negative Log Likelihood Loss

3.9.2.3.

Binary Cross Entropy Loss

3.9.2.4.

3.9.3.

Custom Loss Functions

3.9.3.1.

Implementing Custom Losses

3.9.3.2.

Combining Multiple Losses

Previous

2. PyTorch Fundamentals

Go to top

Next

4. Data Loading and Processing