Blog

Longform notes on building — architecture, trade-offs, and what production taught me.

Series

Governing self-classified documents at scale

How a two-agent LLM system caught underclassified, sensitive financial documents before they became a security incident — and the design decisions that mattered more than the models themselves.

A case study in document classification governance. If you want the transferable lessons without the ML detail, start with the disclosure problem in Part 3 and the rollout metrics in Part 4.

  • Applied ML
  • Agentic systems
  • Security/Governance
  • Model distillation
  • RLHF
Two-agent document classification flow — document save, classifier agent, verifier agent, and escalation to the safer label
  1. Part 1

    The self-classification problem

    A structurally unenforceable policy, systematic underclassification at thousands of documents a day, and why keyword matching was never going to close the gap.

  2. Part 2

    A two-agent system, and the trade-off that shaped everything

    Why one model wasn't enough, how classifier and verifier agents split the work, and the distillation and RLHF loops that kept cost and behavior in check.

  3. Part 3

    The two problems that almost broke this

    Grounding model claims in real document lines, and the paradox where the classification system itself became the disclosure it was meant to prevent.

  4. Part 4

    Rollout, results, and what I'd change today

    Phased deployment across three stages, an 85% reduction in underclassification, and what I'd rebuild differently with today's models.

Series

Building Trust into ML Pipelines: Governance, Audit, and Repeatability

How a low-throughput ML rollup pipeline got trust, audit, and repeatability — patterns that transfer anywhere promoted outputs need a product surface.

A deep dive into one pipeline I built. If you want the pattern without the domain detail, start with the trade-offs in Part 2 and the risk tables in Part 4 and Part 5.

  • Applied ML
  • Pipeline design
  • Orchestration
  • Batch compute
  • Object storage
Four-layer pipeline diagram for holistic ROAS — upstream models, resolution, batch compute, and business surface
  1. Part 1

    When model outputs aren't enough

    Promoted ML outputs without governance — no audit trail, no latest pointer, no repeatability. The systems problem behind holistic ROAS, told without assuming you care about marketing analytics.

  2. Part 2

    Four layers, one contract

    Resolver patterns, immutable taxonomy fields, and submit-time resolution — the orchestration design that turns a pile of promoted runs into one frozen batch input.

  3. Part 3

    What we proved

    Evidence from a full production compute chain — runtimes, artifacts, and the stakeholder moment where a 2.5-hour green run was mistaken for 'done.'

  4. Part 4

    What's hard next

    The remaining risk isn't batch failure — it's operational ambiguity: backfill, validation, rollback, and who owns the run button.

  5. Part 5

    When the blocker moves upstream

    Layer A and B shipped — and the remaining gate turned out to be upstream model promotion, not orchestration code.