vLLM disaggregation benchmark questions NIXL payoff

This is a useful reality check for teams treating disaggregated serving as a default architecture rather than a workload-specific tradeoff.

• –The strongest result is lower inter-token latency, especially in prefill-heavy workloads where separating decode from prompt processing reduces contention.
• –The biggest downside is that KV cache transfer and fixed prefill/decode node splits can hammer throughput and TTFT, especially when long prompts saturate the prefill side.
• –A simple routed setup with independent nodes beat the disaggregated layouts on throughput, which makes plain load balancing look like a stronger baseline than a lot of infra teams assume.
• –The post matters because it tests real serving topologies on AWS EFA instead of repeating the usual theoretical upside of disaggregation.
• –The conclusions are narrow but valuable: if your traffic has low prefix-cache hit rates or short responses, disaggregation can add complexity without delivering the headline win.