An application that calls a model directly knows only what the client library returns: a response and a token count. It does not know whether a cheaper model would have done, why the answer was slow, what the month will cost at this rate or whether the user found the answer useful. LLM observability puts those facts in one place per request and in aggregate, and a routing layer is the natural place to collect them because every request already passes through it.
The unit of observability is the trace. For a routed request it records the signals read from the request, every candidate with its score, price and the rules that favoured or removed it, the target chosen and its confidence, the provider call with latency and tokens, the cost, any fallback and, when it is reported, the outcome. The trace is what makes a decision explainable after the fact - to an engineer debugging a bad answer, to a finance lead asking why the bill moved, to an auditor asking who could have seen a record.
Four metrics carry most of the value: cost per request and per feature, with what the same traffic would have cost on a single frontier model; latency at the median and the tail, per target; success - a response that arrived, parsed and was not refused; and quality, which only the application can report, as feedback the router learns from. Routing makes the first visible in a way a bill cannot, because it knows the counterfactual.
Observability and privacy pull against each other, and the default should be chosen deliberately. This router stores the decision - endpoint, target, status, tokens, cost, signals - and not the text; prompts and answers are logged only when a policy switches it on, and the audit trail is hash-chained so an entry cannot be altered without detection. Metrics and traces export to the tools a team already runs through OpenTelemetry-style telemetry sinks.
