prime-rl gets modular Algorithms layer
Prime Intellect has updated its asynchronous reinforcement learning framework, prime-rl, with a modular algorithms layer that allows running multiple RL algorithms per environment in a single training run. The open-source framework supports algorithms like GRPO and ECHO resolved per environment, enabling mixed objectives and multi-teacher distillation in one run.
By shifting algorithms from global runtime switches to per-environment configurations, prime-rl solves a major pain point in scaling agentic post-training, allowing models to learn from the exact signal each task actually requires instead of a compromised average.
* **Per-Environment Resolution**: Allows training different environments with different objectives (e.g., world-modeling/ECHO for terminal environments vs. GRPO for search/math) in a single run, reducing overhead and maximizing training efficiency.
* **Clean Abstraction**: Researchers can implement new algorithms by subclassing a simple runtime class with setup and scoring hooks without modifying trainer internals.
* **Streamlined Loss Packing**: The trainer receives token-level weight streams instead of direct code execution, allowing samples from different algorithms to pack into the same micro-batch without gradient dilution.
* **External Endpoint Distillation**: Makes multi-teacher On-Policy Distillation (OPD) incredibly straightforward, distilling from different domain-expert teacher models (e.g., coding experts vs. math experts) simultaneously.
DISCOVERED
1h ago
2026-07-06
PUBLISHED
1h ago
2026-07-06
RELEVANCE
AUTHOR
PrimeIntellect