Qwen3.6-35B-A3B KLD sweep spots quant tradeoffs

The main point is that quantization is not a simple “lower bits wins” story. On this model, the author’s read is that FP8 can still trail INT8 on quality, while NVFP4 looks attractive on paper but does not automatically deliver better quality or speed in practice.

• –The benchmark is presented as real GPU work in vLLM, not a synthetic offline proxy, so the comparison is aimed at serving reality rather than theory.
• –KLD is used as a divergence signal, but the post correctly notes that better KLD does not guarantee better task evals for a specific workload.
• –The FP8 vs INT8 comparison reinforces the common serving tradeoff: if the kernel path is mature, a higher-precision format can still be the safer deployment choice.
• –NVFP4 is framed as especially sensitive to implementation details and activation handling, which means “4-bit” alone is not a performance guarantee.
• –For practitioners, the useful takeaway is to pick quantization by end-to-end behavior on your own GPU stack, not by headline bit-width alone.