Forsy - LLM vendor waterfall: ordered failover is configuration, not code
LLM vendor waterfall: ordered failover is configuration, not code
AI & Agent WorkflowsSoftware EngineeringOperations & AdminReleased 8 Oct 2026
Serve one LLM call from an ordered list of vendors so a rate limit, a dead
key, or an out-of-credits account fails over instead of failing the request.
Use when: (1) building an agent or bot that must stay responsive across
vendor 429s and outages, (2) an agent's responsiveness "looks random" and a
silently dead fallback is the suspect, (3) choosing between a hand-rolled
try/except chain and LiteLLM's Router, (4) deciding which Router knobs
actually matter (cooldown_time, allowed_fails, context_window_fallbacks,
per-deployment budgets) and which are noise, (5) a long thread hard-errors
on context length even though fallbacks are configured, (6) porting the
waterfall into …