Airframe

Why Airframe

Observable and documented by default

A team should not have to build its own monitoring or draw its own architecture diagrams. A product's catalog entry gives it a Grafana folder, dashboards and alert rules; its logs, metrics and traces reach them with no setup; and each merge publishes the product's part of a map of the whole organization, which rebuilds itself.

How a product becomes observable and documented

  1. The entry gives it a folder and dashboards

    When a product's catalog entry merges, the operations repository's Grafana root runs in the chain of reconcilers. In the organization's one Grafana Cloud stack it creates the product's folder, which its team can edit and everyone can view, with a service dashboard and an SLO dashboard that open on production, beside the cost dashboard of the FinOps page.

  2. Alert rules come with it

    The same root creates the product's baseline alert rules, with thresholds from the entry's slo: the error budget burning fast or slowly, server errors above 2 percent, latency above the objective or twice it, Cloud SQL near its limits, Pub/Sub messages waiting more than ten minutes, and a dead-letter topic that is not empty. They select production by the env label and email the team's contact point. Today that contact point reaches the maintainer's own accounts, because Grafana Cloud accepts only its own members' addresses and the maintainer's accounts are its only members.

  3. Telemetry flows with no setup

    An application writes JSON logs to its standard output and sends traces and metrics with OTLP to the address the platform gives it; it never holds a Grafana credential. The platform runs one pair of gateways per edition, production and non-production. Traces and metrics go to the edition's OTLP gateway with the service's identity token, and logs travel through Cloud Logging and a Pub/Sub buffer to the logs gateway. The gateways, Grafana Alloy on Cloud Run, check every signal's env label, stamp their edition on it and forward it with a write-only token. The address arrives in the product's Ready output, and the golden path's modules pass it to every service.

  4. Every merge publishes its part of the map

    Every product repository's pipeline includes the catalog-publish component, and so do the catalog's and those of four platform repositories. On a merge request it builds the repository's fragment, its part of the map, with no credential, and fails when the overview or the runbook is missing. After the merge it publishes the fragment under the repository's own prefix: for a product, its service, its API and event specifications, the products it calls, its runbook and its decision records.

  5. The map rebuilds itself

    An aggregator rebuilds the map from the latest fragments right after each publication and every hour, at catalog.projectairframe.dev behind IAP, where everyone in the organization can read it. It takes the list of products from the catalog, skips a broken fragment so the rest still builds, and keeps the last three versions. The map follows each repository's main branch, not what is deployed.

A worked example

This is part of the invoicing product's entry in the demo catalog, and what the platform derives from it. The folder, the dashboards and the alert rules come from the entry alone; signals need the product's projects to be billed, and no demo product is billed today, so the telemetry below is what would flow.

product: invoicing
journey: settle
domain: billing
context: invoicing
team: billing-core
slo: { availability: 99.5, latency_p95_ms: 400 }
  • A folder for the product under settle, billing and invoicing, which billing-core edits and everyone views
  • Dashboards "invoicing service" and "invoicing SLO", opening on prod, beside "invoicing cost"
  • Alert rules with the entry's thresholds: an error budget for 99.5 percent availability, a ticket when p95 latency stays above 400 ms and a page above 800 ms
  • Alerts to billing-core's contact point, which today emails the maintainer's own accounts
  • Traces and metrics that carry invoicing's service name, its domain and its environment, and Cloud resource metrics labeled with the product's path, settle/billing/invoicing/invoicing, with no Grafana credential in the repository
  • A service page for invoicing on catalog.projectairframe.dev, built from the fragment its repository publishes at each merge to main: its overview, its owners, its runbook and its decision records

What makes it true

Every product has its dashboards and alert rules from its entry.
operations' Grafana root, which runs when the catalog changes, creates each catalog product's folder, which its team edits and everyone views, its service and SLO dashboards and its baseline alert rules, with thresholds from the entry's slo; the rules select production by the env label, and a missing series is not an alert.
An alert reaches the product's team, and today that means the maintainer.
Each rule routes by the team label to the team's contact point, which emails the team's members who have a Grafana account, and a page repeats every four hours until it is resolved. Grafana Cloud accepts only its own members' addresses, and today those members are the maintainer's own accounts, so every team's alerts, and the platform's, reach the maintainer.
An application needs no setup and holds no Grafana credential.
The project factory's Ready output carries the address of the gateway of the product's edition, production or non-production, and the golden path's Cloud Run module injects it into every service. Traces and metrics reach the OTLP gateway with the service's identity token, which the gateway's invoker check admits; logs reach the logs gateway through the product project's sink and the Pub/Sub buffer. The gateways hold the edition's write-only token, the only Grafana credential on that side.
Every signal says which edition it came from.
The OTLP gateway keeps a signal's env only when it is valid for its own edition and replaces a missing or foreign value with unknown; the logs gateway keeps a valid env label and otherwise gives the line its perimeter's, so only audit entries stay unknown. Both stamp the edition as perimeter. The golden path's Cloud Run module gives a service's traces and metrics its service name, its domain and its environment, and the platform agent labels each billed product project's Cloud resource metrics with the product's path and its journey and domain.
Each product publishes its own part of the map, and only its own.
The catalog-publish component's validate:fragment job builds the fragment on the merge request with no credential and fails on a missing overview or runbook; on the protected main, release:catalog publishes it with the identity the repository already holds. A product's production apply identity may write only under its product's prefix, and the catalog repository writes as platform-catalog@ under its own. The four platform repositories share one identity, platform-docs@, whose grant covers the whole platform prefix, so nothing stops one of them from writing another's part.
The map rebuilds itself from the latest fragments.
The aggregator, a Cloud Run job, runs when a publishing identity asks after a publication, which is all that identity may do in the project, and hourly on a schedule. It reads the catalog's list of products, skips a broken fragment with its reasons in the run's record and keeps the last three versions. The site behind IAP checks for a new version every minute and keeps serving the last one if it cannot read it.