UsefulLinks
Computer Science
Big Data
Apache Airflow
1. Introduction to Apache Airflow
2. Core Concepts of Airflow
3. Airflow Architecture and Components
4. Setting Up an Airflow Environment
5. Authoring Your First DAG
6. Comprehensive Guide to Operators
7. Managing Workflows with the Airflow UI
8. Scheduling and Triggers
9. Data Sharing and Communication
10. Advanced DAG Authoring Techniques
11. Airflow Providers and Extensibility
12. Testing and Debugging Airflow DAGs
13. Operational Airflow Management
14. Scaling and Production Deployment
15. Airflow Best Practices and Patterns
12.
Testing and Debugging Airflow DAGs
12.1.
Testing Strategies
12.1.1.
Unit Testing Approach
12.1.2.
Integration Testing
12.1.3.
End-to-End Testing
12.2.
Unit Testing DAGs
12.2.1.
Testing DAG Integrity
12.2.2.
Testing Task Dependencies
12.2.3.
Testing Custom Operators
12.2.4.
Testing Hooks and Connections
12.2.5.
Mock Objects and Fixtures
12.3.
CLI Testing Commands
12.3.1.
airflow tasks test Command
12.3.2.
airflow dags test Command
12.3.3.
Command Syntax and Options
12.3.4.
Interpreting Test Results
12.4.
Local Development and Debugging
12.4.1.
Development Environment Setup
12.4.2.
Using Print Statements and Logging
12.4.3.
Isolating Task Failures
12.4.4.
Step-by-Step Debugging
12.5.
DebugExecutor
12.5.1.
Local Debugging Workflow
12.5.2.
DebugExecutor Configuration
12.5.3.
Limitations and Use Cases
12.6.
Common Issues and Troubleshooting
12.6.1.
Import Errors
12.6.2.
Circular Dependencies
12.6.3.
Top-level Code Execution
12.6.4.
Non-idempotent Tasks
12.6.5.
Hardcoded Values
12.6.6.
Memory and Performance Issues
12.7.
Debugging Tools and Techniques
12.7.1.
Log Analysis
12.7.2.
Task Instance Inspection
12.7.3.
Database Query Analysis
12.7.4.
Performance Profiling
Previous
11. Airflow Providers and Extensibility
Go to top
Next
13. Operational Airflow Management