Streaming Data Processing with Apache Kafka and KSQL

  1. Fundamentals of Apache Kafka
    1. Core Concepts
      1. Events and Messages
        1. Structure of a Kafka Message
          1. Message Keys and Values
            1. Message Headers
              1. Message Timestamps
              2. Topics and Partitions
                1. Definition of a Topic
                  1. Partitioning Strategy
                    1. Partition Assignment
                      1. Partition Ordering Guarantees
                      2. Producers and Consumers
                        1. Producer Responsibilities
                          1. Consumer Responsibilities
                            1. Producer Acknowledgments
                              1. Consumer Offsets
                                1. Producer Batching
                                  1. Consumer Polling
                                  2. Brokers and Clusters
                                    1. Role of a Broker
                                      1. Cluster Topology
                                        1. Broker Discovery
                                          1. Cluster Membership
                                          2. Offsets
                                            1. Offset Management
                                              1. Committed vs. Uncommitted Offsets
                                                1. Offset Reset Strategies
                                                2. Consumer Groups
                                                  1. Group Coordination
                                                    1. Load Balancing
                                                      1. Rebalancing Events
                                                        1. Consumer Group Protocol
                                                        2. Log-Structured Storage
                                                          1. Log Segments
                                                            1. Retention Policies
                                                              1. Compaction
                                                                1. Index Files
                                                              2. Kafka Architecture
                                                                1. Role of ZooKeeper and KRaft
                                                                  1. Cluster Metadata Management
                                                                    1. Controller Election
                                                                      1. Transition to KRaft
                                                                        1. KRaft Benefits and Limitations
                                                                        2. Replication and Fault Tolerance
                                                                          1. Replication Factor
                                                                            1. Data Durability
                                                                              1. Consistency Guarantees
                                                                              2. Leader and Follower Replicas
                                                                                1. Leader Election
                                                                                  1. Follower Synchronization
                                                                                    1. Replica Lag Monitoring
                                                                                    2. In-Sync Replicas (ISR)
                                                                                      1. Definition of ISR
                                                                                        1. Impact on Availability
                                                                                          1. ISR Management
                                                                                        2. Kafka Ecosystem Components
                                                                                          1. Kafka Connect for Data Integration
                                                                                            1. Source Connectors
                                                                                              1. Sink Connectors
                                                                                                1. Connector Configuration
                                                                                                  1. Connector Management
                                                                                                  2. Kafka Streams for Client-Side Processing
                                                                                                    1. Stream Processing API
                                                                                                      1. Topology Definition
                                                                                                        1. State Stores
                                                                                                        2. Kafka REST Proxy
                                                                                                          1. RESTful Access to Kafka
                                                                                                            1. Use Cases for REST Proxy
                                                                                                              1. API Endpoints
                                                                                                              2. Schema Registry
                                                                                                                1. Schema Storage
                                                                                                                  1. Schema Validation
                                                                                                                    1. Compatibility Checks
                                                                                                                      1. Schema Evolution

                                                                                                                  Previous

                                                                                                                  1. Introduction to Stream Processing

                                                                                                                  Go to top

                                                                                                                  Next

                                                                                                                  3. Introduction to ksqlDB