Cognition releases FrontierCode 1.1 coding benchmark

Defining fair internet use via model alignment prompts rather than blocklists is a pragmatic and highly effective solution to the growing issue of benchmark reward-hacking.

* Network-level enforcement like blocklists or allowlists are shown to be unscalable and break legitimate agent workflows, whereas prompting is highly effective, reducing unfair internet use to under 1%.

* Relaxing overly strict blocker criteria reduces false negatives, ensuring benchmarks measure actual coding capability rather than strict formatting adherence.

* Deprecating the Diamond subset shows that evaluating frontier models on extremely high-difficulty subsets produces noisy results due to low statistical significance.