OPEN LINK ↗
// 1h agoBENCHMARK RESULT
StarSkirmish Bench v0.2 tests LLMs in StarCraft
[StarSkirmish Bench v0.2](https://starskirmish.com/bench/v0-2/) gives flagship models one hour to write a Protoss bot in C++ before evaluating five-run averages against fixed human-written and demo rosters. GPT-6 Astra leads, while Gemini 4 Argon places third in a close cluster with Claude Opus 5.5 and Claude Fable 5.1.
// ANALYSIS
The benchmark’s real value is exposing long-horizon planning and time allocation, not proving models can play StarCraft at human level.
- –GPT-6 Astra wins only 7% against the strongest human-written bots, leaving a wide capability gap.
- –Practice against top opponents is the strongest predictor of performance, making feedback-driven iteration central to agent quality.
- –Gemini 4 Argon shows notably high run-to-run variance, suggesting inconsistent strategy formation.
- –Fixed rosters and plain win rates make v0.2 easier to compare across model generations than Elo-based scoring.
- –First-party CLI harnesses improve realism but also mix model capability with tooling and orchestration effects.
// TAGS
starskirmish-benchbenchmarkevaluationllmai-codingcoding-agentagent
DISCOVERED
1h ago
2026-10-16
PUBLISHED
3h ago
2026-10-16
RELEVANCE
9/ 10
AUTHOR
__cayenne__
