BEHAVIOR-1K sets template for ARC-AGI-5

Good benchmark design should feel like work, not puzzles, and BEHAVIOR-1K gets closer to that than ARC-style grids ever will. The catch is that once you move into embodied tasks, you inherit simulation fidelity, transfer, and scoring headaches.

• –ARC Prize already pushed ARC-AGI-3 toward interactive reasoning; BEHAVIOR-1K extends that same direction into human-centered robot work.
• –The survey-driven task set is harder to game because it reflects what people actually want robots to do, not synthetic trick questions.
• –Long-horizon chores across many scenes and objects force planning, recovery, and state tracking instead of one-shot pattern matching.
• –Realistic simulation makes the benchmark more meaningful, but also more brittle, expensive, and harder to standardize at scale.
• –If ARC-AGI-5 borrows this philosophy, it should reward useful generalization over leaderboard cleverness.