Apache Airflow

9.

Data Sharing and Communication

9.1.

Cross-Communication (XComs)

9.1.1.

XCom Concept and Purpose

9.1.2.

XCom Data Types and Limitations

9.1.3.

Pushing Data to XComs

9.1.3.1.

9.1.3.2.

Implicit Push from Return Values

9.1.4.

Pulling Data from XComs

9.1.4.1.

Pull by Task ID

9.1.4.2.

9.1.5.

XCom Backend Configuration

9.1.6.

XCom Cleanup and Management

9.1.7.

XCom Security Considerations

9.2.

9.2.1.

TaskFlow Overview and Benefits

9.2.2.

Decorator-based DAG Authoring

9.2.3.

@task Decorator

9.2.4.

Simplified Data Passing

9.2.5.

Type Hints and Data Validation

9.2.6.

TaskFlow vs Traditional Operators

9.2.7.

Multiple Outputs

9.2.8.

TaskFlow with External Systems

9.3.

Airflow Variables

9.3.1.

Variable Storage and Retrieval

9.3.2.

Setting Variables via UI

9.3.3.

Setting Variables via CLI

9.3.4.

Accessing Variables in DAGs

9.3.5.

Variable Templating

9.3.6.

Environment-specific Variables

9.3.7.

Security Considerations for Variables

9.4.

Airflow Connections

9.4.1.

Connection Management Concept

9.4.2.

Connection Components

9.4.2.1.

9.4.2.2.

Connection Type

9.4.2.3.

9.4.2.4.

Schema/Database

9.4.2.5.

Login and Password

9.4.2.6.

Extra Parameters

9.4.3.

Connection Types

9.4.3.1.

Database Connections

9.4.3.2.

Cloud Provider Connections

9.4.3.3.

API Connections

9.4.3.4.

File System Connections

9.4.4.

Managing Connections via UI

9.4.5.

Managing Connections via CLI

9.4.6.

Accessing Connections via Hooks

9.4.7.

Connection Security and Encryption

Previous

8. Scheduling and Triggers

Go to top

Next

10. Advanced DAG Authoring Techniques