Qwen3.6 Benchmarks Favor Plain Dual-GPU Runs

The hot take is that this is less a "find the magic quant" problem and more a "respect the PCIe ceiling" problem. Once inter-GPU traffic becomes part of the decode path, speculative tricks can erase their own gains.

• –Qwen3.6-35B-A3B is the clearer win on this hardware: vLLM NVFP4 with TP2 gives the best balance of prompt throughput and token generation.
• –Qwen3.6-27B dense is more sensitive to backend and quant choice; vLLM beats llama.cpp on raw prompt speed, but token speed and TTFT trade off quickly.
• –The llama.cpp layer-split setups are interesting mainly because they keep performance more balanced, not because they dominate every metric.
• –The failed speculative decoding runs are a useful signal: on 2x 16GB cards, the bottleneck is probably data movement, not model compute.
• –If the goal is practical local use, tuning backend placement and KV/cache strategy matters more than chasing speculative decoding on this class of dual-GPU setup.