TurboQuant sparks llama.cpp KV confusion

TurboQuant is real research, but the community is running into the usual gap between paper benchmark and boring production flag.

• –Google's blog positions TurboQuant as a KV-cache compression win, including 3-bit cache quantization, 6x memory reduction, and up to 8x attention-logit speedups in its tested setup
• –llama.cpp users already have cache quantization via q4_0/q8_0-style types, but TurboQuant-specific KV cache support is not yet a simple, official llama-server path
• –Community forks such as TheTom's and other CUDA/ROCm/Vulkan experiments are moving fast, but reports still include GPU fallback, quality-validation, and backend-coverage caveats
• –For local inference users, this matters because context length, not just model weights, is the VRAM pressure point that decides whether long-context workloads fit on consumer GPUs
• –The near-term story is "watch the PRs and forks," not "drop one flag into production llama-server"