Claude Opus 4.8 doubles GPT-5.5 on FrontierCode

Coding benchmarks are finally growing up, shifting from simple test-passing to the actual, subjective standard of whether a human maintainer would merge the code—and the results show that even our best models are still far from autonomous software engineers.

* The "mergeability" standard is a much higher and more realistic bar than unit-test passing, exposing significant gaps in agent code quality, scope control, and testing practices.

* Claude Opus 4.8's dominance (13.4%) over GPT-5.5 (6.3%) suggests that Anthropic's reasoning and agentic capabilities currently hold a substantial lead in handling complex, multi-step software engineering tasks.

* The extremely low scores across all models on the Diamond tier (all under 14%) highlight that fully autonomous code generation in real-world, production repositories remains an unsolved problem.