Moonshot AI debuts Attention Residuals architecture

Attention Residuals is the first serious rethink of the residual connection in a decade, replacing fixed addition with learned selectivity to prevent context loss in deep architectures. By utilizing Block Attention Residuals, the system maintains hardware efficiency with under 2% latency overhead while allowing models to autonomously organize internal pathways. Scaling experiments show the architecture matches baseline performance with 25% less training compute, marking a foundational step toward more agentic AI reasoning.