Qwen3.5-27B sparks KV cache debate

My read: for Qwen3.5, long context is the point, so I’d protect the usable context window first and treat weight precision as the second lever. If q8 KV cache gets you back above 128K, that is usually the cleaner first experiment before jumping straight to q4 weights.

• –Qwen3.5-27B is natively built for 262,144 tokens and can extend far beyond that, so context budget is a first-class feature rather than a nice-to-have.
• –The model card’s 128K guidance is a strong clue that starving context is more dangerous than shaving a bit of cache precision.
• –llama.cpp already exposes `--cache-type-k` and `--cache-type-v` with `q8_0`, `q4_0`, `q4_1`, and other cache formats, so this is a real tuning axis, not a hack.
• –Weight quantization changes every forward pass; KV quantization mostly pressures long-context recall and decode quality, which makes it the more targeted compromise.
• –Benchmarks on your own workload still matter: code, retrieval, and long reasoning tend to expose cache degradation faster than short chat.