Software EngineeringAI & Agent WorkflowsOpen accessPublished 3 Oct 2026
Use when the user asks how to safely ship an intelligence-affecting change to a production agent — a prompt, tool, harness/scaffold, retrieval, or config change — e.g. "how do we stop shipping agent regressions?", "should our evals block CI?", "how do I roll this agent change out gradually?", "do we need an A/B test or a canary?", "how do we know the change caused the improvement?", or "is dogfooding enough?". Walks the gate ladder — regression evals in CI (blocking vs auditing), clean-attribution A/B design, shadow mode and canaries, feature flags and gradual/rainbow rollouts, online evals after full traffic — with per-gate fit criteria, thresholds, and failure modes lik…