SpectralQuant tops TurboQuant with sparse-key compression
SpectralQuant is an open-source KV cache compression method for LLM inference that targets key vectors, not model weights. The repo claims key cache signal is concentrated in only about 3 to 4% of the head dimension across several model families, so it calibrates once, keeps the informative dimensions, and skips error correction on the rest. The authors say this yields better compression and better quality than TurboQuant, with headline results including a 5.95x compression ratio versus 5.02x for TurboQuant, faster latency, and similar perplexity on their reported benchmarks.
Strong claim, but it reads like a research repo with benchmark-first positioning rather than a productized runtime.
- –The core idea is simple and plausible: exploit spectral sparsity in KV keys to discard most dimensions after calibration.
- –The repo explicitly frames the gain as relative to TurboQuant, so the real question is robustness across more models, prompts, and serving stacks.
- –The reported 15-second calibration and 2.2x latency win are the most practical signals here.
- –The big caveat is that these are author-reported results from a fresh repo, so external replication will matter more than the headline number.
DISCOVERED
52d ago
2026-04-07
PUBLISHED
52d ago
2026-04-07
RELEVANCE
AUTHOR
OmarBessa
