TurboQuant, Attention Residuals slash LLM costs

This is the kind of efficiency research that matters because it attacks both sides of the model bill: inference memory and training compute. TurboQuant looks like the nearer-term product win, while Attention Residuals is the bolder bet on changing how transformers route information.

• –TurboQuant is compelling because it is drop-in: no retraining, no architecture surgery, just less KV-cache pressure and cheaper long-context serving.
• –The H100 speedup is more than a benchmark trophy; it points to memory bandwidth as the real bottleneck in modern LLM inference.
• –Attention Residuals is ambitious but higher risk: learned depth-wise routing could improve scaling, yet it will need independent replications before it becomes standard.
• –The bigger business takeaway is that memory movement and information routing are becoming as important as model size when teams try to cut serving and training costs.