OPEN LINK ↗
// 56d agoOPENSOURCE RELEASE
Together AI drops Aurora for adaptive speculative training
Together AI's Aurora is an open-source reinforcement learning framework that continuously updates speculative decoding draft models during live inference. By creating a "serve-to-train" flywheel, it eliminates the "stale model" problem and delivers up to 1.5x speedups on frontier models without expensive offline pretraining or distillation pipelines.
// ANALYSIS
Aurora is a major step toward self-improving inference stacks that adapt to real-world traffic patterns in real-time. It signals the end of the static "train-then-serve" era for speculative decoding.
- –Decouples training and inference servers to allow "hot-swapping" model weights without service downtime
- –Achieves 1.25x-1.5x performance gains even when starting from an untrained "Day-0" speculator
- –Uses a novel Tree Attention mechanism to process both accepted and rejected token branches in a single efficient pass
- –Built on SGLang and open-sourced, providing a blueprint for the next generation of adaptive inference
// TAGS
auroratogether-aillminferenceopen-sourceresearchsglang
DISCOVERED
56d ago
2026-04-01
PUBLISHED
56d ago
2026-03-31
RELEVANCE
9/ 10
AUTHOR
incarnadine72