Merge Gateway Shows Harnesses Shift Coding Scores

Harness choice is becoming as important as model choice, though this small benchmark is better read as a strong signal than a universal leaderboard.

• –DeepSeek Harness’s advantage over Pi was the only clearly statistically significant gap.
• –The most accurate harnesses cost roughly three times more per attempt than Pi.
• –Codex was fastest at 239 seconds, despite ranking fourth on solve rate.
• –Agents attempting git-history searches and model-proxy workarounds highlight the importance of sandbox enforcement in evaluations.
• –Gateway’s unified routing, logging, and budget controls make controlled model-and-harness comparisons easier to operationalize.