Vector Search and Embeddings

  1. Data Representation: Embeddings
    1. What are Embeddings
      1. Definition of Vector Embeddings
        1. The Goal of Capturing Semantic Meaning
          1. From Unstructured Data to Numerical Vectors
            1. Embedding Space Structure and Properties
              1. Dimensionality Considerations
              2. How Embeddings are Created
                1. The Role of Machine Learning
                  1. Neural Network Architectures for Embeddings
                    1. Training Objectives and Loss Functions
                      1. Contrastive Learning
                        1. Metric Learning
                        2. Training Paradigms
                          1. Supervised Training
                            1. Unsupervised Training
                              1. Self-Supervised Learning
                                1. Transfer Learning
                                2. Pre-trained vs Fine-tuned Models
                                  1. Advantages of Pre-trained Models
                                    1. Limitations of General Models
                                      1. Domain Adaptation through Fine-tuning
                                        1. Custom Model Training
                                      2. Text Embeddings
                                        1. Word-Level Embeddings
                                          1. Word2Vec
                                            1. Skip-gram Architecture
                                              1. CBOW Architecture
                                                1. Training Process
                                                2. GloVe
                                                  1. Global Matrix Factorization
                                                    1. Co-occurrence Statistics
                                                    2. FastText
                                                      1. Subword Information
                                                        1. Out-of-Vocabulary Handling
                                                      2. Sentence and Document Embeddings
                                                        1. Sentence-BERT
                                                          1. Siamese Network Architecture
                                                            1. Fine-tuning for Sentence Similarity
                                                            2. Universal Sentence Encoder
                                                              1. Transformer Architecture
                                                                1. Multi-task Training
                                                                2. Modern Transformer-based Models
                                                                  1. BERT and Variants
                                                                    1. RoBERTa
                                                                      1. DistilBERT
                                                                    2. Contextual Embeddings
                                                                      1. Context-Dependent Representations
                                                                        1. Handling Polysemy
                                                                          1. Dynamic Embeddings
                                                                        2. Image Embeddings
                                                                          1. Convolutional Neural Networks
                                                                            1. Feature Extraction Layers
                                                                              1. Pooling and Dimensionality Reduction
                                                                                1. Transfer Learning from ImageNet
                                                                                2. Vision Transformers
                                                                                  1. Patch-based Processing
                                                                                    1. Self-Attention Mechanisms
                                                                                      1. Positional Encodings
                                                                                      2. Specialized Image Embedding Models
                                                                                        1. ResNet Features
                                                                                          1. EfficientNet
                                                                                            1. CLIP Image Encoder
                                                                                          2. Audio Embeddings
                                                                                            1. Spectrogram-Based Embeddings
                                                                                              1. Mel-frequency Cepstral Coefficients
                                                                                                1. Short-time Fourier Transform
                                                                                                2. Deep Audio Models
                                                                                                  1. Audio Transformers
                                                                                                    1. Wav2Vec
                                                                                                      1. Audio Neural Networks
                                                                                                      2. Speech vs Music Embeddings
                                                                                                      3. Multimodal Embeddings
                                                                                                        1. Joint Text-Image Embeddings
                                                                                                          1. Shared Embedding Spaces
                                                                                                            1. Cross-modal Retrieval
                                                                                                            2. CLIP Model
                                                                                                              1. Contrastive Pre-training
                                                                                                                1. Zero-shot Classification
                                                                                                                  1. Text-Image Alignment
                                                                                                                  2. Other Multimodal Approaches
                                                                                                                    1. ALIGN
                                                                                                                      1. DALL-E Embeddings
                                                                                                                    2. Properties of Good Embedding Spaces
                                                                                                                      1. Proximity and Similarity Preservation
                                                                                                                        1. Similar Items Cluster Together
                                                                                                                          1. Distance Metrics Correlation
                                                                                                                          2. Analogies and Relationships
                                                                                                                            1. Vector Arithmetic for Semantic Relationships
                                                                                                                              1. Linear Relationships in Embedding Space
                                                                                                                              2. Robustness and Generalization
                                                                                                                                1. Noise Tolerance
                                                                                                                                  1. Variation Handling
                                                                                                                                    1. Domain Transfer

                                                                                                                                Previous

                                                                                                                                2. Mathematical Foundations: Vectors and Vector Spaces

                                                                                                                                Go to top

                                                                                                                                Next

                                                                                                                                4. Similarity Metrics and Distance Functions