Standards that hold: rules a machine can check
A platform with N teams survives on conventions a machine can check. Names, tags, owners, enforced at merge, visible in the UI.
The previous lesson built the chassis. This one keeps fifteen teams from shipping chaos into it, and the answer is boring on purpose: a naming rule, a tag rule, and a CI job that says no.
Two levers, and why a machine pulls them
- Custom operators. The platform team ships tested, documented building blocks (a dbt operator, a Snowflake operator, a Tableau operator), so feature teams write SQL instead of reinventing orchestration plumbing.
- Naming and tagging rules, checked by CI. Every DAG is called
team__domain__typeand carries the tags owner, domain and criticality. A commit that doesn't conform gets its merge blocked.
The second one is this lesson. It sounds bureaucratic until you open an Airflow UI with three hundred DAGs in it and try to answer "which of these is ours, and which one matters at 3am?"
The rules, at a glance
Two commits, two fates
owner:marketing, domain:growth, criticality:high. Anyone can route an incident in five seconds.What the Airflow UI looks like once the rule holds
aggregate and publish daily sales KPIs
ingest marketing leads from source systems
build and refresh the revenue 360 mart
run data quality checks on critical datasets
Sort three hundred DAGs by name and they group by team for free. Filter one tag and you have a team's whole estate. That's what a naming rule buys: findability at 3am.
The job that says no
The check is a short script in the shared CI templates. It loads every DAG the way Airflow would, lists every broken rule, and fails the pipeline if there's one:
# ci/check_ import re, sys from airflow.dag_ standards. py, run on every merge request models import DagBag NAME = re.compile(r"^[a-z0-9]+__ ) # team_[a-z0-9_ ]+_ _ [a-z]+$" _ REQUIRED = {"owner", "domain", "criticality"} bag = DagBag("dags/", include_domain_ _ type examples=False) errors = [f"{path}: import error" for path in bag.import_ for dag_id, dag in bag.errors] dags. if not NAME.items( ): match( errors.dag_ id): append(f"{dag_id}: name must be team_ if missing := REQUIRED - {t.split(":")[0] for t in dag.tags}: errors._ )domain_ _ type" append(f"{dag_id}: missing tags {sorted( print("\n".missing)}" )join( sys.exit(1 if errors else 0) # non-zero = merge blockederrors) or "all DAGs conform")

