Airflow: when it runs, in what order, what if it breaks
A Python file says what should happen and when. Airflow does the rest, every day, and tells you when it doesn't.
Thomas's first week
Thomas wrote a cron job. One line on a small server in the corner of the office: 0 6 * * * dbt run. It worked for eleven days.
- Day 12. Snowflake dropped the connection for four seconds. dbt failed, cron shrugged, and nothing rebuilt. Nobody found out until 11am.
- Day 19. The loader ran late, so dbt built on yesterday's rows. Everything reported success. The dashboards were quietly wrong for a day and a half.
- Day 26. Somebody asked which model broke on the 14th. The logs had rolled over. There were no logs.
- Day 33. The little server needed a reboot. Nobody knew it existed, so nobody restarted the cron. Three days of no data.
None of that is a new problem. Retrying a flaky task, refusing to run before the data has landed, keeping the history of what ran and when: this is exactly what Airflow was built to do, and Thomas gets all of it by writing a Python file instead of a cron line. The fourth day is a different problem, and it belongs to Astronomer, the next lesson.
People say the two names in the same breath, which makes them sound like alternatives. They're not. One is the tool, the other is where the tool lives:
Where Airflow sits
Read it bottom to top: Astronomer keeps Airflow alive, Airflow reads your files, your files say what should happen and when. And to the right, at arm's length, the things that do the actual work.
What Airflow does for us
In our case, Airflow runs Alice's dbt project every morning and watches what happens. You write a Python file that says run this, then this, then this, every day at six, and Airflow takes care of it from then on, whether Thomas is at his desk or on holiday. It runs dbt as the platform's service account, through a connection stored in Airflow, never with anyone's laptop credentials.
Three decisions, and only three:
The four things Thomas stops building himself
Every one of them was a day in that first week.
Schedule
Run dbt every morning at 06:00, no human watching.
Retry
Snowflake slow, connection dropped? Try again instead of failing the morning.
Alert
Tell Thomas the moment something truly fails, not a manager at 9am.
Coordinate
Run dbt after ingestion and before the dashboard refresh. In order, every time.
Before Airflow, and after
What the same platform looks like on both sides of the migration.
✗ Without a shared orchestrator
✓ With Airflow on Astronomer
astro dev start runs the whole platform on your laptop first.One thing is still missing from this picture. Airflow is software, and software has to run somewhere. That's the next lesson.

