2026 survey maps RL algorithms for reasoning LLMs

This is the clearest single resource for practitioners trying to navigate the crowded and fast-moving RL-for-reasoning space — the field has gone from "not enough algorithms" to "too many to evaluate."

• –The central finding: learned critic networks (PPO-style) appear unnecessary for fine-tuned LLMs; simpler group-relative baselines work just as well with much lower memory overhead
• –Standard deviation normalization — used almost everywhere — is identified as consistently hurting asymptotic performance, a subtle failure mode most practitioners haven't noticed
• –Token-level loss aggregation prevents verbosity bias, a detail that matters enormously for deployment but is rarely discussed in papers
• –Four open problems remain unresolved: credit assignment beyond pass/fail rewards, sample efficiency, handling unsolvable hard problems, and generalizing beyond math/code domains
• –ScaleRL's 400K+ GPU-hour validation adds rare empirical grounding to a field otherwise dominated by theoretical analysis