Build my path

e9a5f13branch(deploy):git switch production

Production

See every request, know what it cost, and catch regressions before users do.

Tracing, budgets and regression tests for model features. You instrument a working app so a bad answer can be traced to the prompt, the retrieval or the tool that caused it.

for
Builders with something working who need it to stay working.
time
4 weeks at 3 hours a week
not for
Projects that are still ideas. Build first, then come here.
foundationsbuildautomatedeploylead

You will ship

A monitored assistant with traces, a cost dashboard, alerts and a test suite that runs on every change.

Prerequisites

  • A working AI feature, or the App Engineer project
  • Comfortable reading logs
  • Access to a place to deploy

e9a5e01plan:six commits to ship

The branch, commit by commit.

0/6 commits done · saved in this browser only

  1. commit 1One trace per request

    Every model call, retrieval and tool call becomes a span with timing and token counts.

    Where this happens: /lessons/agent-trace#try
  2. commit 2Cost per request

    Turn token counts into money per request and per user action.

    Where this happens: /lessons/agent-trace#cost
  3. commit 3Latency budget

    Set a budget per step and find which step breaks it.

    Where this happens: /courses/ship-and-see-it#m2
  4. commit 4Regression suite in CI

    Your test set runs on every prompt or model change and blocks merges that score worse.

    Where this happens: /courses/evals-before-vibes#m4
  5. commit 5Alerts that mean something

    Alert on error rate, cost spikes and refusal spikes, not on every slow call.

    Where this happens: /courses/ship-and-see-it#m4
  6. commit 6Ship the dashboard

    One page anyone on the team can read to know if the feature is healthy.

    Where this happens: /projects#trace-dashboard

e9a5c02docs:courses on this branch

Courses.

e9a5d03ship:capstone

Capstone: Trace and cost dashboard

One page showing traces, cost per run, latency per step and refusal rate.

Tests it must pass

  • every request has one trace id
  • cost per run under budget
  • no raw user text stored
See the demo build
span.ts ts
span.set({ model: m, tokens_in, tokens_out, cost_usd: price(m, tokens_in, tokens_out) });span.set({ user_text_len: text.length }); // shape, not content