The K2 Team has released K2-Horizon, a "radically open" fleet of six LLMs ranging from 0.9B to 375B parameters that provide unprecedented training transparency.
The K2 Team has launched K2-Horizon, a "radically open" fleet of six LLMs—including dense models (0.9B–32B) and Mixture-of-Experts (MoE) models up to the 375B-A23B flagship. By strictly adhering to LLM360 standards, the release provides researchers with not just final model weights, but also comprehensive training data, logs, intermediate checkpoints, and the full training/RL framework. The fleet introduces several technical innovations, including Mixture-of-Value Attention (MoVA) for enhanced reasoning and Uno decoding adapters for faster inference.
K2-Horizon sets a new bar for transparency in the open-source AI ecosystem by providing the complete "recipe" for reproducibility rather than just the final artifact.
• The use of MoVA and softplus gating in the MoE architectures is a sophisticated approach to sparse modeling that aims to maximize reasoning performance while maintaining parameter efficiency.
• Releasing Uno decoding adapters is a high-value addition for developers aiming to optimize inference speed without sacrificing model quality or needing a separate draft model.
• This release heavily emphasizes scientific transparency and reproducibility, offering an invaluable resource for researchers studying the dynamics of large-scale pre-training, data pipelines, and sparsity.
DISCOVERED
10h ago
2026-11-18
PUBLISHED
11h ago
2026-11-18
RELEVANCE
AUTHOR
papersdatacode