Failover that actually fails over

A failover path you have never exercised is a hypothesis, not a feature. The uncomfortable truth about multi-provider inference is that the hard part is rarely the routing logic. It is everything the routing logic assumed: that weights are already resident on the standby, that the tokenizer version matches, that quota exists on the other side, that the client's timeout is longer than a cold start.

Health is not a ping

A provider that answers a TCP handshake in forty milliseconds can still be returning malformed completions. We treat a route as healthy only when it passes four checks in sequence: the endpoint accepts a connection, it returns a completion for a canary prompt, that completion parses, and its latency sits inside the route's declared budget.

Any one of those failing marks the route degraded rather than dead. Degraded routes keep receiving a small trickle of traffic so we can tell when they recover, instead of waiting for a human to flip them back.

Budget the cold start, not the average

When a route drops, the standby is usually cold. If you size your failover on steady-state latency you will trade one outage for another. Pin a minimum warm replica on at least one alternate provider for anything with a user waiting at the other end.

Practise on purpose

Mauvi can drain a provider on a schedule so the path gets exercised during business hours, with people watching, rather than at 03:00 on a Sunday. If that idea makes you nervous, that is the signal: the path is not ready.