ARC-AGI-3 sinks frontier models below 1%

This looks less like a model leaderboard and more like a stress test for the whole agent stack. If ARC-AGI-3 sticks, the meaningful comparison will be model plus harness plus search, not raw model scores alone.

• –Interactive benchmarks expose how much performance depends on scaffolding: memory, state tracking, action selection, and exploration strategy matter as much as the base model.
• –Efficiency-based scoring changes the optimization target; thrashy token-heavy reasoning can become a liability instead of a feature.
• –The right leaderboard treatment is probably a separate agentic track, because static text evals and turn-based environments measure different capabilities.
• –The reported RL + graph-search result is the most interesting signal here: algorithmic search may beat brute-force parameter scaling on this class of task.
• –Open-weight evaluation will only be meaningful if the harness is standardized, otherwise quantization, prompting, and tool wrappers will swamp the score.