AutoRL automates live RL model training
Researchers introduced AutoRL, a multi-agent framework that provides continuous process-level supervision and dynamic interventions during live reinforcement learning runs. In experiments, it sustained weeks of unattended training while lifting Qwen3-4B's AIME 2024 score from 46.67% to 66.67%.
Static reinforcement learning recipes frequently collapse or plateau without constant human oversight, turning manual researcher intervention into the primary bottleneck for frontier reasoning models. AutoRL demonstrates that coordinating specialized agents to monitor telemetry and hot-patch training dynamics in flight unlocks scalable, unattended self-improvement.
- –Three-tier agent architecture: Uses Perceptors to track loss spikes and evaluation plateaus, Proposers to formulate interventions, and an Orchestrator to apply live hot updates or checkpoint rollbacks.
- –Significant reasoning gains: Propelled Qwen3-4B's AIME 2024 benchmark from 46.67% to 66.67% and AIME 2025 from 45.00% to 56.67%, outperforming both static GRPO recipes and general-agent baselines across nine benchmarks.
- –14.7 days of unattended training: Autonomously resolved 11 runtime execution failures during extended coding agent runs, boosting SWE-bench pass@1 by 12.84%.
- –Limits of agentic RSI: While the multi-agent loop excelled at hyperparameter adjustments, looping penalties, and failure recovery, the authors noted agents still failed to invent fundamentally novel learning algorithms.
DISCOVERED
31m ago
2026-11-16
PUBLISHED
1h ago
2026-11-16
RELEVANCE
AUTHOR
omarsar0