AutoEnvScaling Turns Agents Into Environment Builders
AutoEnvScaling treats RL environment design as a terminal task: a proposer agent reads solver failures, builds and validates new Harbor environments, and uses fresh solver rollouts to guide the next iteration. The paper reports up to 25× higher valid-environment rates, 75% lower cost per valid environment, and measurable Terminal-Bench gains.
AutoEnvScaling makes the data flywheel executable instead of manually curated, bringing recursive self-improvement closer to a practical training workflow.
- –Terminal access lets proposers write, run, debug, and validate complete environments rather than generate brittle task descriptions.
- –Solver feedback is central: environments built from actual failures outperform static pools and prompt-only generation.
- –GPT-5.6-sol-generated environments improved Qwen3.5-35B-A3B from 34.6 to 38.9 on Terminal-Bench 2.1.
- –Qwen3.6-35B-A3B improved from 41.1 to 53.3 on Terminal-Bench 2.1 when acting as both proposer and solver.
- –Terminal-Bench 4.0 gains remained modest, showing that benchmark generalization and robust RSI are still open problems.
DISCOVERED
1h ago
2026-10-15
PUBLISHED
1h ago
2026-10-15
RELEVANCE
AUTHOR
arnaudmercier