TurboQuant.cpp lands 1-bit KV cache

The interesting part is not just the compression ratio, but that the author is showing a working inference engine with real model outputs and measurable quality deltas. It is still early, though: the repo is solidly benchmark-driven and the most mature path appears to be Apple Silicon/CPU-first rather than a broadly deployed production stack.

• –The strongest claim is byte-identical generation on Qwen3.5-35B-A3B MoE, which is a better sanity check than a single perplexity number
• –The quality story looks credible on the reported Gemma 3 4B test: FP16 KV at 35.99 vs 1-bit K + Q4 V at 36.00 is effectively flat
• –This is infrastructure, not a consumer app: the value is in fitting longer contexts and larger models into less memory
• –The repo being built from scratch, with tests and GGUF support, makes it more interesting than a paper implementation alone
• –The main caveat is scope: the current validation set is small, and the headline performance is still hardware- and model-dependent