Apache Spark

  1. Structured Streaming
    1. Streaming Fundamentals
      1. Streaming Model
        1. Unbounded Table Concept
          1. Micro-Batch Processing
            1. Continuous Processing
            2. Time Semantics
              1. Processing Time
                1. Event Time
                  1. Ingestion Time
                  2. Watermarking
                    1. Late Data Handling
                      1. Watermark Configuration
                        1. State Cleanup
                        2. Output Modes
                          1. Append Mode
                            1. Complete Mode
                              1. Update Mode
                            2. Streaming API Components
                              1. Input Sources
                                1. File Source
                                  1. Directory Monitoring
                                    1. File Format Support
                                    2. Kafka Source
                                      1. Topic Subscription
                                        1. Offset Management
                                          1. Consumer Configuration
                                          2. Socket Source
                                            1. TCP Connection
                                              1. Text Stream Processing
                                              2. Rate Source
                                                1. Synthetic Data Generation
                                                  1. Testing Applications
                                                2. Output Sinks
                                                  1. File Sink
                                                    1. Partitioning Strategies
                                                      1. File Format Options
                                                      2. Kafka Sink
                                                        1. Producer Configuration
                                                          1. Serialization Options
                                                          2. Console Sink
                                                            1. Debug Output
                                                              1. Development Testing
                                                              2. Foreach Sink
                                                                1. Custom Output Logic
                                                                  1. External System Integration
                                                                2. Query Management
                                                                  1. Query Lifecycle
                                                                    1. Trigger Configuration
                                                                      1. Processing Time Triggers
                                                                        1. Once Triggers
                                                                          1. Continuous Triggers
                                                                          2. Query Monitoring
                                                                        2. Windowing Operations
                                                                          1. Window Types
                                                                            1. Tumbling Windows
                                                                              1. Fixed-Size Windows
                                                                                1. Non-Overlapping Intervals
                                                                                2. Sliding Windows
                                                                                  1. Overlapping Windows
                                                                                    1. Slide Duration Configuration
                                                                                    2. Session Windows
                                                                                      1. Gap-Based Grouping
                                                                                        1. Dynamic Window Sizing
                                                                                      2. Window Functions
                                                                                        1. Aggregation in Windows
                                                                                          1. Window Specifications
                                                                                            1. Time-Based Grouping
                                                                                          2. State Management
                                                                                            1. Stateful Operations
                                                                                              1. State Store Implementation
                                                                                                1. State Partitioning
                                                                                                  1. State Evolution
                                                                                                  2. Checkpointing
                                                                                                    1. Checkpoint Configuration
                                                                                                      1. Recovery Mechanisms
                                                                                                        1. Checkpoint Storage
                                                                                                        2. Fault Tolerance
                                                                                                          1. Exactly-Once Semantics
                                                                                                            1. At-Least-Once Processing
                                                                                                              1. Failure Recovery

                                                                                                          Previous

                                                                                                          4. Spark SQL and Structured APIs

                                                                                                          Go to top

                                                                                                          Next

                                                                                                          6. Machine Learning with MLlib