Qwen3.5 MoE hits 9.5 tok/s on Strix Halo

Cool experiment, but this reads more like a topology lesson than a throughput win. On a sparse MoE model, EP only pays if cross-node traffic stays tame, and consumer APUs usually expose the pain fast.

• –The official Qwen3.5 card describes the model as a 122B-parameter MoE with 256 experts and 8 routed + 1 shared active per token, so routing overhead is baked into the problem.
• –Qwen's own serving guidance leans on SGLang or vLLM with 8-way tensor parallel, which suggests the default high-performance path is still a mature serving stack, not bespoke cluster choreography.
• –Strix Halo's 128GB unified memory is what makes these experiments possible, but unified memory does not erase bandwidth and interconnect ceilings.
• –One commenter in the thread says a single 128GB Strix Halo can already hit roughly 23-25 tok/s on the same model/quant, so 9.5 tok/s across two machines looks more like an early prototype than a scaling win.
• –Before jumping to custom ROCm kernels, I'd profile whether the bottleneck is routing, memory copies, or scheduler overhead; that answer will tell you whether EP, pipeline parallelism, or a dense-model baseline is the real move.