Apache Airflow

  1. Data Sharing and Communication
    1. Cross-Communication (XComs)
      1. XCom Concept and Purpose
        1. XCom Data Types and Limitations
          1. Pushing Data to XComs
            1. Explicit Push
              1. Implicit Push from Return Values
              2. Pulling Data from XComs
                1. Pull by Task ID
                  1. Pull by Key
                  2. XCom Backend Configuration
                    1. XCom Cleanup and Management
                      1. XCom Security Considerations
                      2. TaskFlow API
                        1. TaskFlow Overview and Benefits
                          1. Decorator-based DAG Authoring
                            1. @task Decorator
                              1. Simplified Data Passing
                                1. Type Hints and Data Validation
                                  1. TaskFlow vs Traditional Operators
                                    1. Multiple Outputs
                                      1. TaskFlow with External Systems
                                      2. Airflow Variables
                                        1. Variable Storage and Retrieval
                                          1. Setting Variables via UI
                                            1. Setting Variables via CLI
                                              1. Accessing Variables in DAGs
                                                1. Variable Templating
                                                  1. Environment-specific Variables
                                                    1. Security Considerations for Variables
                                                    2. Airflow Connections
                                                      1. Connection Management Concept
                                                        1. Connection Components
                                                          1. Connection ID
                                                            1. Connection Type
                                                              1. Host and Port
                                                                1. Schema/Database
                                                                  1. Login and Password
                                                                    1. Extra Parameters
                                                                    2. Connection Types
                                                                      1. Database Connections
                                                                        1. Cloud Provider Connections
                                                                          1. API Connections
                                                                            1. File System Connections
                                                                            2. Managing Connections via UI
                                                                              1. Managing Connections via CLI
                                                                                1. Accessing Connections via Hooks
                                                                                  1. Connection Security and Encryption

                                                                                Previous

                                                                                8. Scheduling and Triggers

                                                                                Go to top

                                                                                Next

                                                                                10. Advanced DAG Authoring Techniques