Prompt optimization proves coin-flip unreliable in compound AI

The sharp takeaway is that prompt optimization is becoming an overused default in agent stacks: if your system already sits near the zero-shot ceiling, tuning burns time without buying much.

• –Across 72 runs on Claude Haiku, 49% of optimized prompts scored below zero-shot; Amazon Nova Lite fared even worse
• –Joint interaction effects were never significant, which weakens the case for expensive end-to-end prompt co-optimization
• –The gains showed up mainly when the task exposed a format the model could produce but did not naturally choose
• –The proposed $80 ANOVA pre-test and 10-minute headroom test are useful as a cheap stop/go filter before you invest in optimization
• –This is a strong paper for teams building multi-agent systems with TextGrad, DSPy, or similar tooling: optimize selectively, not by reflex