Qwen 3.6 hits performance walls with TurboQuant

TurboQuant's ambitious KV cache compression is proving too heavy for Qwen 3.6's already lean architecture, turning a memory win into a performance loss.

• –Qwen 3.6's Mixture-of-Experts (35B-A3B) design already features a highly efficient KV cache, reducing the margin for gains from additional quantization.
• –Users on r/LocalLLaMA report generation speeds as low as 36 t/s on high-end dual-GPU setups, compared to 50+ t/s using standard implementations.
• –The "sweet spot" appears to be restricted to the Value cache (-ctv) only, as quantizing the Key cache (-ctk) introduces disproportionate quality and speed trade-offs.
• –The high computational cost of the required Hadamard transforms (random rotations) can lead to a 15-30x end-to-end performance drop if not using specialized kernels.
• –As core llama.cpp begins integrating similar features like AttnRot, specialized forks like TheTom/llama-cpp-turboquant may struggle to stay relevant without upstreaming their optimizations.