GPT-6.2 Astra, Sol flop on SimpleBench
SimpleBench updated its leaderboard with OpenAI's newly released GPT-6.2 Astra and Sol models, revealing unexpected performance regressions across common-sense reasoning and trick-question evaluations. Both models dropped below the 83.7% human baseline, with Sol 6.2 plunging 13 points to 30th place while rival models Claude Fable 5.2 and Gemini 4 Neon maintain top ranks above 94%.
OpenAI's push to optimize 6.2 for agentic coding and slide generation appears to have come at the direct expense of basic semantic precision and common-sense logic.
- –Severe regressions across the lineup: Sol 6.2 collapsed 13 points to 69.9% (30th place), while GPT-6.2 Astra Pro dropped from 86.5% to 79.2%, falling below the 83.7% unspecialized human baseline.
- –Overthinking penalty: SimpleBench deliberately penalizes models on trick questions and spatial reasoning, suggesting OpenAI's updated post-training may suffer from reasoning drift on straightforward prompts.
- –Frontier gap widens: Anthropic's Claude Fable 5.2 (94.9%) and Google's Gemini 4 Neon (94.4%) demonstrate that frontier scale does not have to compromise everyday sanity checks.
- –Implications for developers: Teams utilizing GPT-6.2 models for classification, structured extraction, or plain-text reasoning should rigorously test for regressions before upgrading from 6.1.
DISCOVERED
4h ago
2026-11-21
PUBLISHED
7h ago
2026-11-21
RELEVANCE
AUTHOR
algoth1