Observability

AI observability reconstructs one request — model, tokens, retrieve, tools — not a green dashboard. Eval scores the product; traces explain a failure.

Traces are the request tree, logs are events on that tree, metrics are fleet aggregates — join them with a trace id, and sample the expensive and failing calls.

Track input, cached, reasoning, and output tokens plus TTFT and total time — then dollars per successful task, with alerts on loops and cache busts.

Nested spans for retrieve, each model call, and each tool — a waterfall that shows which stage failed, without dumping secret payloads.

Tag every span with prompt, model, and retriever versions. Eval traces match prod shape so you can replay an incident, then flip the flag.