Deep Learning and Neural Networks

  1. Convolutional Neural Networks (CNNs)
    1. Motivation for CNNs
      1. Challenges with High-Dimensional Data
        1. Limitations of MLPs for Image Data
          1. Spatial Structure in Images
            1. Translation Invariance
              1. Local Connectivity Principles
                1. Parameter Sharing Benefits
                2. Core Components of CNNs
                  1. The Convolutional Layer
                    1. Convolution Operation
                      1. Filters and Kernels
                        1. Filter Size Selection
                          1. Number of Filters
                            1. Learnable Parameters
                            2. Stride and Padding
                              1. Stride Definition and Effects
                                1. Padding Types
                                  1. Valid Padding
                                    1. Same Padding
                                      1. Causal Padding
                                    2. Feature Maps
                                      1. Interpretation and Visualization
                                        1. Depth and Spatial Dimensions
                                        2. Receptive Fields
                                          1. Local Connectivity
                                            1. Effective Receptive Field
                                          2. The Pooling Layer
                                            1. Purpose of Pooling
                                              1. Downsampling Operations
                                                1. Max Pooling
                                                  1. Operation and Effects
                                                    1. Translation Invariance
                                                    2. Average Pooling
                                                      1. Operation and Effects
                                                        1. Smooth Downsampling
                                                        2. Global Pooling
                                                          1. Global Average Pooling
                                                            1. Global Max Pooling
                                                            2. Adaptive Pooling
                                                            3. Fully Connected Layers
                                                              1. Role in Classification
                                                                1. Flattening Feature Maps
                                                                  1. Parameter Count Considerations
                                                                  2. Activation Functions in CNNs
                                                                    1. ReLU in Convolutional Layers
                                                                      1. Softmax in Output Layers
                                                                    2. CNN Architectures
                                                                      1. Classic Architectures
                                                                        1. LeNet-5
                                                                          1. Structure and Innovations
                                                                            1. Historical Significance
                                                                            2. AlexNet
                                                                              1. Deep Architecture
                                                                                1. Use of ReLU and Dropout
                                                                                  1. GPU Implementation
                                                                                  2. VGGNet
                                                                                    1. Deep and Uniform Architecture
                                                                                      1. Small Filter Sizes
                                                                                      2. GoogLeNet (Inception)
                                                                                        1. Inception Modules
                                                                                          1. Multi-Scale Processing
                                                                                            1. 1x1 Convolutions
                                                                                          2. Modern Architectures
                                                                                            1. ResNet (Residual Networks)
                                                                                              1. Residual Connections
                                                                                                1. Deep Architectures
                                                                                                  1. Batch Normalization Integration
                                                                                                  2. DenseNet
                                                                                                    1. Dense Connections
                                                                                                      1. Feature Reuse
                                                                                                      2. EfficientNet
                                                                                                        1. Compound Scaling
                                                                                                          1. Neural Architecture Search
                                                                                                      3. Advanced CNN Concepts
                                                                                                        1. Dilated Convolutions
                                                                                                          1. Atrous Convolutions
                                                                                                            1. Receptive Field Expansion
                                                                                                            2. Separable Convolutions
                                                                                                              1. Depthwise Separable Convolutions
                                                                                                                1. Parameter Reduction
                                                                                                                2. Grouped Convolutions
                                                                                                                  1. Channel Grouping
                                                                                                                    1. Computational Efficiency
                                                                                                                    2. Transposed Convolutions
                                                                                                                      1. Upsampling Operations
                                                                                                                        1. Deconvolution Terminology
                                                                                                                      2. Computer Vision Applications
                                                                                                                        1. Image Classification
                                                                                                                          1. Single-Label Classification
                                                                                                                            1. Multi-Label Classification
                                                                                                                              1. Fine-Grained Classification
                                                                                                                              2. Object Detection
                                                                                                                                1. Bounding Box Prediction
                                                                                                                                  1. Two-Stage Detectors
                                                                                                                                    1. R-CNN Family
                                                                                                                                      1. Region Proposal Networks
                                                                                                                                      2. One-Stage Detectors
                                                                                                                                        1. YOLO
                                                                                                                                          1. SSD
                                                                                                                                        2. Semantic Segmentation
                                                                                                                                          1. Pixel-wise Classification
                                                                                                                                            1. Fully Convolutional Networks
                                                                                                                                              1. U-Net Architecture
                                                                                                                                              2. Instance Segmentation
                                                                                                                                                1. Object Detection and Segmentation
                                                                                                                                                  1. Mask R-CNN
                                                                                                                                                  2. Image Generation
                                                                                                                                                    1. Generative Models in Vision
                                                                                                                                                      1. Style Transfer
                                                                                                                                                    2. Transfer Learning with CNNs
                                                                                                                                                      1. Pre-trained Models
                                                                                                                                                        1. ImageNet Pre-training
                                                                                                                                                          1. Model Zoos
                                                                                                                                                          2. Fine-Tuning Strategies
                                                                                                                                                            1. Layer Freezing
                                                                                                                                                              1. Learning Rate Adjustment
                                                                                                                                                              2. Feature Extraction
                                                                                                                                                                1. Fixed Feature Extractor
                                                                                                                                                                  1. Bottleneck Features
                                                                                                                                                                  2. Domain Adaptation
                                                                                                                                                                    1. Applications and Benefits

                                                                                                                                                                  Previous

                                                                                                                                                                  4. Practical Considerations for Training

                                                                                                                                                                  Go to top

                                                                                                                                                                  Next

                                                                                                                                                                  6. Recurrent Neural Networks (RNNs)