Astronomer: managed Airflow, and what it saves you
Somebody has to keep Airflow itself alive. Astronomer is the answer to that question, and it costs you one command.
The problem: Airflow has to live somewhere
Airflow is software, and software has to run somewhere. Left to yourself that means a scheduler process, a metadata database, workers, an API server for the UI, and someone patching all of it. Thomas's day-33 problem was exactly this: a machine nobody was watching.
Astronomer runs that for you. Same Airflow, same Python files, same UI, except the servers are somebody else's problem. It's the trade you already made with Snowflake: you could install a database on a machine you patch yourself, and instead you never think about the machine.
What you get in practice:
- It's always up. Scheduler, database, workers and API server are hosted, monitored, patched and upgraded. No reboot takes your pipelines down.
- It scales on its own. Twenty DAGs one month, two hundred the next, and you don't size a server for it.
- Local development is one command.
astro dev startruns Airflow in Docker on your laptop, on the same Runtime image as production, so you test a DAG before anyone sees it. - Deployment is one command.
astro deployships your project, and in this platform GitLab CI runs it for you after the merge. - Support and guardrails. Sensible defaults, an upgrade path that doesn't involve a migration weekend, and people to call when Airflow itself misbehaves.
Everything in this guide runs on Astronomer, and nothing you learn here would change if it didn't.
The four pieces you actually touch
Astronomer's offer comes as four named pieces, and you'll hear all four names:
Astro CLI
Local development and deploys. One command spins up Airflow on your laptop with Docker, and DAG changes apply as you save.
Astro
The hosted platform, on AWS, GCP or Azure. Each environment is a Deployment, with auto-scaling workers and no infrastructure to maintain.
Astro Runtime
The Airflow Docker image Astronomer builds, tests and versions, with common providers pre-installed and a published support lifecycle.
Astro Observe
Data observability on top of Airflow: lineage, and freshness or delivery SLAs on the data products a business relies on.
Installing it, and running Airflow on your laptop
Docker Desktop has to be running first, because the CLI starts the whole platform as containers. Then it's four commands:
# 1. install the CLI brew install astro # macOS # Linux: curl -sSL install.astronomer. # Windows: winget install -e --id Astronomer.io | sudo bash -s Astro astro version # 2. create a project (dags/, include/, requirements.txt, Dockerfile) mkdir pizzeria-airflow && cd pizzeria-airflow astro dev init # 3. start it locally: every Airflow component + a Postgres metadata DB astro dev start # → http:/ # drop a .py file into dags/ and it appears, no restart needed # 4. the ones you'll use daily astro dev logs -f # follow the Airflow logs astro dev restart # after changing requirements./ localhost: 8080 (login details print in the terminal) txt, packages.txt or the Dockerfile astro dev bash # a shell inside the container astro dev stop # give the laptop its RAM back
Adding a Python package (say astronomer-cosmos for the dbt lesson) means one line in requirements.txt and an astro dev restart. When it works locally, astro deploy sends the same project to the hosted Deployment.
Hibernation: switch a whole environment off
A Deployment with Development Mode turned on can be hibernated. Scheduler, API server and workers scale to zero and the Deployment stops consuming compute; its configuration is kept, and it wakes up as it was. While it sleeps, nothing runs: no DAGs, no UI, no deploys, and a task still running when it falls asleep is marked failed.
That matters because most environments aren't needed most of the time:
- Dev environments overnight and at weekends. A dev Deployment that only runs while people are working can sleep two thirds of the week.
- On a schedule you set. Define wake hours in the Astro UI (calendar or cron, in UTC) and Astronomer applies them without anyone remembering.
- Or on demand. Override the schedule from the UI, the API or
astro deployment wake-upwhen someone needs to test outside the window. - Spin environments up and down freely. A Deployment per team or per experiment gets cheap when the idle ones sleep, so you create the environment you need instead of sharing one.
What you'd be operating without it
Here are the parts behind that first list, the ones you'd otherwise run yourself:
The knobs you still get to turn
Astronomer runs these four for you. The one knob you still choose is how the workers are housed: the executor.
The executor: how tasks actually run
astro dev start uses. Astro Deployments don't offer it.The real choice is isolation. Celery and the Astro executor share each worker across many tasks (fast, but a heavy task can starve the small ones unless you split them into worker queues), Kubernetes gives each task its own pod and resource limits (isolated, but every task pays startup time). A dbt run that needs 8 GB shouldn't share a worker with fifty tiny sensors.
CeleryExecutor · one shared worker
KubernetesExecutor · one pod per task
Why KubernetesExecutor is popular on data platforms: a dbt build, a Spark submit and a tiny sensor have wildly different appetites, and one pod per task means each declares its own CPU and memory. The trade is a few seconds of pod startup per task. Here's its whole life:
That's the machine: a scheduler that watches the clock, workers that do the work, an executor that decides how they're housed, and Astronomer running all of it. Next lesson: the tasks themselves.

