Priors Finds Widespread Benchmark Saturation

Benchmark saturation is becoming an infrastructure problem for AI development: leaderboard gains can look meaningful while saying little about real capability. Priors makes the issue visible by cataloging benchmark headroom, spread, recency, and model coverage.

• –Saturated benchmarks compress frontier-model differences into misleadingly small score gaps.
• –Developers should prioritize held-out, private, and task-based evaluations over familiar public tests.
• –Agentic, coding, and long-horizon benchmarks may offer more useful differentiation than static accuracy exams.
• –Priors’ quality scoring helps teams filter benchmark portfolios instead of treating every score as equally informative.