ARC-AGI-3 Public Games Aren’t Eval Scores

ARC-AGI-3 is deliberately moving benchmark credibility behind private holdouts, making public-game optimization a poor proxy for general intelligence.

• –The 25 public environments demonstrate the format and basic mechanics.
• –Private and semi-private environments test broader, out-of-distribution generalization.
• –Public scores can be inflated through task-specific harnesses, human replays, or targeted training.
• –Developers should use the public set for integration testing and agent iteration, not leaderboard claims.
• –The distinction highlights how easily benchmark results become misleading when test data is exposed.