Apache Airflow

  1. Airflow Providers and Extensibility
    1. Provider System Overview
      1. Provider Architecture
        1. Provider Package Structure
          1. Provider Versioning
            1. Provider Installation
            2. Core Provider Packages
              1. Amazon (AWS) Provider
                1. S3 Operators and Sensors
                  1. EC2 Operators
                    1. RDS Operators
                      1. Lambda Operators
                      2. Google Cloud (GCP) Provider
                        1. BigQuery Operators
                          1. Cloud Storage Operators
                            1. Compute Engine Operators
                              1. Dataflow Operators
                              2. Microsoft Azure Provider
                                1. Blob Storage Operators
                                  1. Data Factory Operators
                                    1. Container Instances
                                    2. Database Providers
                                      1. PostgreSQL Provider
                                        1. MySQL Provider
                                          1. SQLite Provider
                                        2. Creating Custom Components
                                          1. Custom Operators
                                            1. Inheriting from BaseOperator
                                              1. Implementing Execute Method
                                                1. Parameter Validation
                                                  1. Template Fields
                                                  2. Custom Hooks
                                                    1. Hook Base Classes
                                                      1. Connection Management
                                                        1. API Integration Patterns
                                                        2. Custom Sensors
                                                          1. Sensor Implementation
                                                            1. Poke vs Reschedule Logic
                                                              1. Custom Sensor Patterns
                                                              2. Custom Timetables
                                                                1. Timetable Interface
                                                                  1. Custom Scheduling Logic
                                                                2. Plugin Development
                                                                  1. Plugin Architecture
                                                                    1. Plugin Structure
                                                                      1. Installing Custom Plugins
                                                                        1. Plugin Distribution
                                                                        2. Community Contributions
                                                                          1. Contributing to Providers
                                                                            1. Provider Development Guidelines
                                                                              1. Testing Provider Components

                                                                            Previous

                                                                            10. Advanced DAG Authoring Techniques

                                                                            Go to top

                                                                            Next

                                                                            12. Testing and Debugging Airflow DAGs