OPEN LINK ↗
// 45d agoBENCHMARK RESULT
fdash triples vLLM inference speed on Qwen 3.5
A developer benchmarked vLLM's speculative decoding methods on Qwen3.5-27B, finding the new fdash proposer nearly triples generation speed to 125 tokens per second. However, fdash currently lacks compatibility with 8-bit KV cache compression, demanding significantly more VRAM than native MTP alternatives.
// ANALYSIS
The speed gains from fdash are staggering, but its heavy memory tax keeps it out of reach for smaller GPU setups.
- –fdash achieved 124.96 TPS compared to the baseline 46.57 TPS without speculation, proving it as a top-tier decoding method for local inference
- –Qwen 3.5's native Multi-Token Prediction (MTP) is slower (84.57 TPS) but supports FP8 KV caching, making it the practical choice for VRAM-constrained environments
- –The lack of FP8 KV cache support for fdash forces users to choose between maximum throughput and memory efficiency until vLLM expands compatibility
// TAGS
vllmqwen-3.5llminferencegpubenchmark
DISCOVERED
45d ago
2026-04-12
PUBLISHED
46d ago
2026-04-12
RELEVANCE
8/ 10
AUTHOR
Sticking_to_Decaf