quant.cpp claims lossless 4-bit KV compression

If the benchmark holds up independently, this is a real memory breakthrough rather than another speed-vs-quality tradeoff. The catch is that the post is self-reported and the community is already pushing back on how novel the underlying KV quantization story really is.

• –The main value prop is context length, not raw throughput: the repo explicitly says to use llama.cpp for speed and quant.cpp for fitting more context in less memory.
• –The implementation angle is unusually practical: standard GGUF loading, pure C, zero dependencies, and a single-header embed path lower adoption friction.
• –The benchmark claims are strong, but they need outside replication before anyone should treat “0.0% PPL delta” as settled.
• –A Reddit comment notes llama.cpp already supports separate K/V quantization types, so the differentiator here is likely the specific scheme and reported quality, not the existence of KV quantization itself.