AGI-Pilled Bench adds Riemann task, redesign
Adam Holter updated AGI-Pilled Bench with a site redesign and a new milestone evaluating whether frontier models can produce Lean-verified proofs of the Generalized Riemann Hypothesis. Evaluated across 42 reasoning configurations, models like GPT-6 Astra estimated a median of 34 years before AI achieves the mathematical breakthrough.
Benchmarking what models believe about their own capability horizons exposes a stark gap between frontier reasoning benchmarks and autonomous research reality.
- –GPT-6 Astra's 34-year forecast (targeting 2060) reveals that models remain deeply conservative about formalizing unproven mathematics despite high reasoning scores.
- –The redesign tracks 17 models across 42 reasoning configurations using native CLI harnesses and OpenRouter to isolate test-time compute effects.
- –Across previous tests, zero models recognized recent 2026 mathematical and cryptographic milestones had already occurred, highlighting ongoing temporal blindness.
- –The project demonstrates how self-prediction prompts serve as useful meta-evals for calibrating model humility versus overconfidence.
DISCOVERED
11m ago
2026-11-20
PUBLISHED
33m ago
2026-11-20
RELEVANCE
AUTHOR
AdamHoltererer