Four layers, one contract
Resolver patterns, immutable taxonomy fields, and submit-time resolution — the orchestration design that turns a pile of promoted runs into one frozen batch input.
Part 1 named the blind spot: governance, not math. This post is the orchestration design — how I closed it.
HROAS is not a new model. It is orchestration and rollup over outputs that already exist — national MMM (with DTC and commerce as distinct slots), geo-level MMM, and brand-equity components. When Econ promotes a model run, the pipeline should resolve the latest promoted inputs, compute holistic ROAS in one batch job, and surface a stable latest answer with a full audit trail.
Manual handoff. Know which runs are live. Copy object-store paths into scripts. Re-run when someone remembers. Tribal knowledge.
Promote a model → resolver picks inputs → batch computes → latest updates. Every run auditable. No path archaeology.
I split that story into four layers so each piece has exactly one job.
Each layer, one responsibility
Layer A — Upstream model jobs
Time-series MMM jobs write promoted outputs to their own tables and stores. Three families matter for HROAS: national MMM (DTC and commerce tracked separately), geo/spatial MMM, and brand-equity runs whose sibling components share a training timestamp.
Layer B — Promotion, resolution, audit
Turns "a pile of promoted model runs" into the exact frozen input for one HROAS computation — and records that decision forever.
Layer C — Batch compute
One job runs five routines in sequence: spend/effect summary, model overview, response curves, efficient frontier, marginal decision data. Standardized JSON in; artifacts and a success marker out.
Layer D — Outputs and frontend
One object-store layout per client. Frontend reads a latest pointer — never a run ID. Chart data and a "which models?" panel for the business surface.
The middle layer is the new intellectual work. Layers A and C largely existed in pieces; Layer B is what makes the handoff automatic and auditable instead of manual. Upstream jobs train or score models; HROAS never retrains — it consumes promoted outputs and rolls them into one governed surface.
Four immutable fields, stamped at birth
The resolver cannot work if slots drift silently. I needed a small, rigid contract between Econ config and database rows — written once at run creation, never edited in place.
Four columns, all immutable after the job row exists:
| Field | Role |
|---|---|
| Dependent variable type | Revenue vs non-revenue HROAS implementations can run in parallel |
| Model type | National DTC, national commerce, geo, or brand equity — DTC and commerce are distinct, not collapsed |
| Model scope | Which client or scope string this row belongs to |
| Regional scope | Geographic partition the run covers |
Econ already authors a JSON config that drives every model run. I extended that config — no shadow mapping table. The job writes the four fields to its row when the run is created; the resolver reads them directly.
Latest wins. Siblings travel together.
Layer B is a declarative registry per client plus a resolver that runs at submit time — before Batch starts.
What the resolver does, in plain language:
- Read the registry — which slots this client needs filled for a complete HROAS run
- For each slot, find the latest client-facing run on main — never a feature branch, never trust a sticky flag alone as the unique selector
- For brand equity, pull the full sibling set anchored on a shared training timestamp — components trained together stay together
- Snapshot the result to an audit table — "what did HROAS use on day X?" is a query, not archaeology
- Hand Batch a frozen JSON payload — compute is pure; it does not re-resolve
Triggers are toggle event + manual button. No cron. HROAS should re-run when models meaningfully change, not every night because we can.
Layer B implementation awaits a final alignment pass with data engineering and the domain team — but the design is locked. I did not want Batch shipping while the trust contract was still fuzzy.
Calls I made — and one I almost got wrong
These are the decisions I locked in so the system stays debuggable:
Resolve at submit time; Batch stays dumb.
I pushed back when a teammate suggested embedding resolution inside the job. Replay would get harder, audit blurrier, and "what did we actually run?" would depend on batch logs instead of a queryable snapshot.
Latest-wins ordering, not promotion flags alone.
I almost shipped with a boolean client_facing field as the selector. A data review caught three rows flagged true on the same slot — ORDER BY date DESC LIMIT 1 on main is explicit; a sticky flag is not.
Runtime clone of the existing Python stack, not a dedicated Batch image.
We spent two weeks on a dedicated image design before I killed it. Ops surface area wasn't worth it; package drift is real but guardable with image-level pins.
Layer D read path before Layer B is finished.
I argued for this in a planning doc when the default plan was "no frontend until orchestration ships." Months of blank UI while compute was already provable wasn't acceptable.
Config UI: engineering builds it, Econ fills it.
Light front end, heavy validation. The registry is a product surface for economists, not a ticket queue for engineering.
Part 3 is evidence: what a full production run actually looked like, and what the numbers do not prove.
Hindsight
I'd invest in resolver unit tests before writing orchestration code. I designed the test matrix in Part 4's risk table while Layer B was still gated — right in hindsight, but it felt backwards at the time. I'm still chewing on validation strictness: block on taxonomy mismatch is safer, warn ships faster, and I don't have enough production incidents yet to know which failure mode hurts more.