Merge Gateway Evals De-Risks Model Swaps

This is the practical missing layer in model switching: offline evals establish a baseline, while shadow traffic reveals how candidates behave against messy production workloads.

• –Uses the same Gateway path, credentials, restrictions, and guardrails as production
• –Per-session shadowing preserves multi-turn context for realistic agent comparisons
• –Cost and latency visibility turns model migration into an operational decision, not a benchmark debate
• –Confidence intervals help expose when a “better” score is based on too few test cases
• –The main risk is still evaluation quality: weak task suites can make a carefully measured migration confidently wrong