Closing the Improvement Loop

Use when the user asks how to close the loop between evals, production data, and shipping — "how do offline and online evals fit together?", "should this eval block the deploy?", "when do we A/B test instead of eval?", "how do production failures become regression tests?", "what's our eval cadence?" — or when specifying the end-to-end quality system for an LLM/agent product. Walks the three-layer closed loop as one system — offline evals gate releases, online evals read production, A/B/n experiments decide on business outcomes — with each layer's blind spots named and a fixed shipping cadence. Not for monitoring mechanics — tracing, alerting, dashboards, SLOs (that is mon…