Terminal-Bench 4.0 Cuts Noise, Retires Saturated Tasks

This is the unglamorous maintenance agent benchmarks need to remain credible. The catch is that a cleaner exam is still a new exam, requiring fresh baselines and careful reporting.

• –A flat eight-hour timeout reduces failures caused by arbitrary per-task limits.
• –Anthropic found resource configuration alone could shift Terminal-Bench scores by six percentage points, underscoring the value of calibration. [Research](https://www.anthropic.com/engineering/infrastructure-noise)
• –Removing tasks solved consistently by frontier systems keeps the leaderboard discriminative.
• –Developers should record the benchmark version, agent harness, effort setting, and resource limits when comparing results.
• –Because Terminal-Bench measures complete agent systems—not just base models—it is especially useful for evaluating real coding workflows. [Homepage](https://www.tbench.ai/)