AI and Creativity

  1. Core Technologies for Generative AI
    1. Foundational Concepts in Generative Models
      1. Probabilistic Modeling
        1. Probability Distributions
          1. Gaussian Distributions
            1. Categorical Distributions
              1. Mixture Models
              2. Sampling Methods
                1. Monte Carlo Sampling
                  1. Markov Chain Monte Carlo
                    1. Importance Sampling
                    2. Maximum Likelihood Estimation
                      1. Parameter Learning
                        1. Model Fitting
                          1. Overfitting Prevention
                        2. Latent Space Representation
                          1. Dimensionality Reduction
                            1. Principal Component Analysis
                              1. t-SNE
                                1. UMAP
                                2. Interpolation in Latent Space
                                  1. Linear Interpolation
                                    1. Spherical Interpolation
                                      1. Semantic Interpolation
                                      2. Disentangled Representations
                                        1. Factor Separation
                                          1. Controllable Generation
                                            1. Interpretability
                                          2. Training Data and Distribution
                                            1. Data Collection and Curation
                                              1. Web Scraping
                                                1. Dataset Assembly
                                                  1. Quality Control
                                                  2. Data Preprocessing
                                                    1. Normalization
                                                      1. Augmentation
                                                        1. Filtering
                                                        2. Data Bias and Limitations
                                                          1. Selection Bias
                                                            1. Representation Gaps
                                                              1. Mitigation Strategies
                                                          2. Generative Adversarial Networks
                                                            1. Core Architecture
                                                              1. The Generator
                                                                1. Network Structure
                                                                  1. Deconvolutional Layers
                                                                    1. Upsampling Techniques
                                                                      1. Skip Connections
                                                                      2. Output Generation
                                                                        1. Noise Input Processing
                                                                          1. Feature Map Generation
                                                                            1. Final Output Layer
                                                                          2. The Discriminator
                                                                            1. Network Structure
                                                                              1. Convolutional Layers
                                                                                1. Feature Extraction
                                                                                  1. Classification Head
                                                                                  2. Real vs Fake Classification
                                                                                    1. Binary Classification
                                                                                      1. Probability Scoring
                                                                                        1. Decision Boundaries
                                                                                    2. The Adversarial Training Process
                                                                                      1. Game Theory Foundation
                                                                                        1. Minimax Game
                                                                                          1. Nash Equilibrium
                                                                                            1. Zero-Sum Competition
                                                                                            2. Loss Functions
                                                                                              1. Generator Loss
                                                                                                1. Discriminator Loss
                                                                                                  1. Alternative Loss Formulations
                                                                                                  2. Training Instability and Solutions
                                                                                                    1. Mode Collapse
                                                                                                      1. Vanishing Gradients
                                                                                                        1. Training Techniques
                                                                                                          1. Progressive Growing
                                                                                                            1. Spectral Normalization
                                                                                                              1. Gradient Penalty
                                                                                                          2. Key GAN Variants
                                                                                                            1. StyleGAN
                                                                                                              1. Style-Based Generation
                                                                                                                1. Style Vectors
                                                                                                                  1. AdaIN Layers
                                                                                                                    1. Progressive Architecture
                                                                                                                    2. Applications in Portraits and Art
                                                                                                                      1. High-Resolution Faces
                                                                                                                        1. Artistic Style Transfer
                                                                                                                          1. Identity Manipulation
                                                                                                                        2. CycleGAN
                                                                                                                          1. Image-to-Image Translation
                                                                                                                            1. Domain Transfer
                                                                                                                              1. Style Conversion
                                                                                                                                1. Content Preservation
                                                                                                                                2. Unpaired Data Training
                                                                                                                                  1. Cycle Consistency Loss
                                                                                                                                    1. Bidirectional Translation
                                                                                                                                      1. Domain Adaptation
                                                                                                                                    2. Conditional GANs
                                                                                                                                      1. Conditioning on Labels or Text
                                                                                                                                        1. Class-Conditional Generation
                                                                                                                                          1. Text-to-Image Synthesis
                                                                                                                                            1. Attribute Control
                                                                                                                                            2. Applications in Controlled Generation
                                                                                                                                              1. Targeted Output
                                                                                                                                                1. User-Specified Constraints
                                                                                                                                                  1. Interactive Generation
                                                                                                                                                2. Progressive GANs
                                                                                                                                                  1. Incremental Resolution Training
                                                                                                                                                    1. Stability Improvements
                                                                                                                                                      1. High-Quality Output
                                                                                                                                                  2. Transformer Models
                                                                                                                                                    1. The Transformer Architecture
                                                                                                                                                      1. Self-Attention Mechanism
                                                                                                                                                        1. Query-Key-Value Framework
                                                                                                                                                          1. Attention Scores and Weights
                                                                                                                                                            1. Dot-Product Attention
                                                                                                                                                              1. Scaled Attention
                                                                                                                                                                1. Softmax Normalization
                                                                                                                                                                2. Multi-Head Attention
                                                                                                                                                                  1. Parallel Attention Heads
                                                                                                                                                                    1. Concatenation and Projection
                                                                                                                                                                      1. Representation Diversity
                                                                                                                                                                    2. Positional Encoding
                                                                                                                                                                      1. Sinusoidal Encoding
                                                                                                                                                                        1. Learned Positional Embeddings
                                                                                                                                                                          1. Relative Position Encoding
                                                                                                                                                                          2. Encoder-Decoder Structure
                                                                                                                                                                            1. Encoder Functionality
                                                                                                                                                                              1. Layer Stacking
                                                                                                                                                                                1. Residual Connections
                                                                                                                                                                                  1. Layer Normalization
                                                                                                                                                                                  2. Decoder Functionality
                                                                                                                                                                                    1. Masked Self-Attention
                                                                                                                                                                                      1. Cross-Attention
                                                                                                                                                                                        1. Autoregressive Generation
                                                                                                                                                                                      2. Feed-Forward Networks
                                                                                                                                                                                        1. Position-wise Processing
                                                                                                                                                                                          1. Non-linear Transformations
                                                                                                                                                                                            1. Dimensionality Changes
                                                                                                                                                                                          2. Large Language Models
                                                                                                                                                                                            1. Model Scaling
                                                                                                                                                                                              1. Parameter Count Growth
                                                                                                                                                                                                1. Computational Requirements
                                                                                                                                                                                                  1. Emergent Abilities
                                                                                                                                                                                                  2. Pre-training and Fine-tuning
                                                                                                                                                                                                    1. Unsupervised Pre-training
                                                                                                                                                                                                      1. Next Token Prediction
                                                                                                                                                                                                        1. Masked Language Modeling
                                                                                                                                                                                                          1. Large-Scale Datasets
                                                                                                                                                                                                          2. Transfer Learning
                                                                                                                                                                                                            1. Knowledge Transfer
                                                                                                                                                                                                              1. Task Adaptation
                                                                                                                                                                                                                1. Few-Shot Learning
                                                                                                                                                                                                                2. Domain Adaptation
                                                                                                                                                                                                                  1. Specialized Fine-tuning
                                                                                                                                                                                                                    1. Domain-Specific Data
                                                                                                                                                                                                                      1. Performance Optimization
                                                                                                                                                                                                                    2. Prompt Engineering
                                                                                                                                                                                                                      1. Prompt Design Strategies
                                                                                                                                                                                                                        1. Zero-Shot Prompting
                                                                                                                                                                                                                          1. Few-Shot Prompting
                                                                                                                                                                                                                            1. Chain-of-Thought Prompting
                                                                                                                                                                                                                            2. Prompt Tuning and Optimization
                                                                                                                                                                                                                              1. Soft Prompts
                                                                                                                                                                                                                                1. Template Learning
                                                                                                                                                                                                                              2. Notable LLM Architectures
                                                                                                                                                                                                                                1. GPT Series
                                                                                                                                                                                                                                  1. BERT and Variants
                                                                                                                                                                                                                                    1. T5 and Text-to-Text Models
                                                                                                                                                                                                                                2. Diffusion Models
                                                                                                                                                                                                                                  1. Theoretical Foundation
                                                                                                                                                                                                                                    1. Stochastic Differential Equations
                                                                                                                                                                                                                                      1. Reverse-Time Processes
                                                                                                                                                                                                                                        1. Score-Based Models
                                                                                                                                                                                                                                        2. The Forward Process
                                                                                                                                                                                                                                          1. Noise Schedules
                                                                                                                                                                                                                                            1. Linear Schedules
                                                                                                                                                                                                                                              1. Cosine Schedules
                                                                                                                                                                                                                                                1. Learned Schedules
                                                                                                                                                                                                                                                2. Progressive Corruption
                                                                                                                                                                                                                                                  1. Gaussian Noise Addition
                                                                                                                                                                                                                                                    1. Variance Scheduling
                                                                                                                                                                                                                                                      1. Information Destruction
                                                                                                                                                                                                                                                    2. The Reverse Process
                                                                                                                                                                                                                                                      1. Denoising Steps
                                                                                                                                                                                                                                                        1. Iterative Refinement
                                                                                                                                                                                                                                                          1. Step Size Control
                                                                                                                                                                                                                                                            1. Convergence Criteria
                                                                                                                                                                                                                                                            2. Sampling Techniques
                                                                                                                                                                                                                                                              1. DDPM Sampling
                                                                                                                                                                                                                                                                1. DDIM Sampling
                                                                                                                                                                                                                                                                  1. Accelerated Sampling
                                                                                                                                                                                                                                                                  2. Neural Network Parameterization
                                                                                                                                                                                                                                                                    1. U-Net Architecture
                                                                                                                                                                                                                                                                      1. Time Conditioning
                                                                                                                                                                                                                                                                        1. Noise Prediction
                                                                                                                                                                                                                                                                      2. Conditioning and Guidance
                                                                                                                                                                                                                                                                        1. Text-to-Image Generation
                                                                                                                                                                                                                                                                          1. Text Embedding Integration
                                                                                                                                                                                                                                                                            1. CLIP Embeddings
                                                                                                                                                                                                                                                                              1. Cross-Attention Mechanisms
                                                                                                                                                                                                                                                                                1. Semantic Alignment
                                                                                                                                                                                                                                                                                2. Classifier-Free Guidance
                                                                                                                                                                                                                                                                                  1. Guidance Scaling
                                                                                                                                                                                                                                                                                    1. Unconditional Training
                                                                                                                                                                                                                                                                                      1. Quality-Diversity Trade-off
                                                                                                                                                                                                                                                                                    2. Image-to-Image Translation
                                                                                                                                                                                                                                                                                      1. Conditional Inputs
                                                                                                                                                                                                                                                                                        1. Inpainting Applications
                                                                                                                                                                                                                                                                                          1. Style Transfer
                                                                                                                                                                                                                                                                                        2. Advanced Diffusion Techniques
                                                                                                                                                                                                                                                                                          1. Latent Diffusion Models
                                                                                                                                                                                                                                                                                            1. Cascaded Diffusion
                                                                                                                                                                                                                                                                                              1. Continuous-Time Models
                                                                                                                                                                                                                                                                                            2. Other Key Architectures
                                                                                                                                                                                                                                                                                              1. Variational Autoencoders
                                                                                                                                                                                                                                                                                                1. Encoder-Decoder Structure
                                                                                                                                                                                                                                                                                                  1. Probabilistic Encoding
                                                                                                                                                                                                                                                                                                    1. Latent Variable Sampling
                                                                                                                                                                                                                                                                                                      1. Reconstruction Decoding
                                                                                                                                                                                                                                                                                                      2. Latent Variable Modeling
                                                                                                                                                                                                                                                                                                        1. Prior Distributions
                                                                                                                                                                                                                                                                                                          1. Posterior Approximation
                                                                                                                                                                                                                                                                                                            1. KL Divergence Regularization
                                                                                                                                                                                                                                                                                                            2. Applications in Generation
                                                                                                                                                                                                                                                                                                              1. Continuous Latent Space
                                                                                                                                                                                                                                                                                                                1. Interpolation Capabilities
                                                                                                                                                                                                                                                                                                                  1. Controllable Generation
                                                                                                                                                                                                                                                                                                                2. Recurrent Neural Networks
                                                                                                                                                                                                                                                                                                                  1. Sequence Modeling
                                                                                                                                                                                                                                                                                                                    1. Temporal Dependencies
                                                                                                                                                                                                                                                                                                                      1. Hidden State Evolution
                                                                                                                                                                                                                                                                                                                        1. Variable-Length Sequences
                                                                                                                                                                                                                                                                                                                        2. Long Short-Term Memory
                                                                                                                                                                                                                                                                                                                          1. Memory Cells and Gates
                                                                                                                                                                                                                                                                                                                            1. Forget Gate
                                                                                                                                                                                                                                                                                                                              1. Input Gate
                                                                                                                                                                                                                                                                                                                                1. Output Gate
                                                                                                                                                                                                                                                                                                                                2. Applications in Sequential Data
                                                                                                                                                                                                                                                                                                                                  1. Text Generation
                                                                                                                                                                                                                                                                                                                                    1. Music Composition
                                                                                                                                                                                                                                                                                                                                      1. Time Series Modeling
                                                                                                                                                                                                                                                                                                                                    2. Gated Recurrent Units
                                                                                                                                                                                                                                                                                                                                      1. Simplified Architecture
                                                                                                                                                                                                                                                                                                                                        1. Reset and Update Gates
                                                                                                                                                                                                                                                                                                                                          1. Computational Efficiency
                                                                                                                                                                                                                                                                                                                                        2. Autoregressive Models
                                                                                                                                                                                                                                                                                                                                          1. Sequential Generation
                                                                                                                                                                                                                                                                                                                                            1. Conditional Probability Modeling
                                                                                                                                                                                                                                                                                                                                              1. Causal Masking
                                                                                                                                                                                                                                                                                                                                              2. Flow-Based Models
                                                                                                                                                                                                                                                                                                                                                1. Normalizing Flows
                                                                                                                                                                                                                                                                                                                                                  1. Invertible Transformations
                                                                                                                                                                                                                                                                                                                                                    1. Exact Likelihood Computation

                                                                                                                                                                                                                                                                                                                                                Previous

                                                                                                                                                                                                                                                                                                                                                1. Foundations of AI and Creativity

                                                                                                                                                                                                                                                                                                                                                Go to top

                                                                                                                                                                                                                                                                                                                                                Next

                                                                                                                                                                                                                                                                                                                                                3. Applications in Creative Domains