llama.cpp users debate MoE bottlenecks

The useful takeaway is that there is no single bottleneck: prompt processing, token generation, expert placement, batch size, cache behavior, and PCIe topology all change the answer.

• –CPU-resident MoE experts can make RAM bandwidth matter, especially when selected experts are computed on CPU rather than copied to GPU.
• –When prompt processing offloads CPU-held weights back to GPU, PCIe bandwidth and main-GPU placement can become the painful limit.
• –ik_llama.cpp is relevant because it explicitly targets CPU/CUDA hybrid performance, fused MoE operations, tensor overrides, and graph split modes.
• –The Xeon upgrade may help CPU-side MoE and AVX-512 kernels, but fewer cores and weaker PCIe topology could hurt multi-GPU pipeline or transfer-heavy runs.
• –This is benchmark territory: `llama-bench` or sweep-style testing with the actual models, batch sizes, and split modes will beat theoretical bandwidth math.