Challenge: Airflow 2
Three situations where a DAG that looks fine misbehaves in production.
The Analytics Engineer’s Field GuideAirflow 2: writing DAGsChallenge: Airflow 2
Three situations where a DAG that looks fine misbehaves in production.
Twenty SqlSensor tasks and twenty S3KeySensor tasks each wait up to two hours every morning. Thomas wants the waiting to tie up as few worker slots as possible. What's the best setup?
process_country.expand(country=countries) runs for 11 countries with retries=2. FR fails twice on a blip, then succeeds; DE fails on all three tries. What's the state of the run?
pick_mode chooses a full refresh on Mondays, using datetime.now(). Monday's run failed on a Snowflake outage, and on Tuesday Thomas clears the whole run. What happens, and what's the fix?
Finish this set to count it as done.