Qwen3.5-122B-A10B hits 80 t/s on RTX Pro 6000

This is the kind of datapoint local inference builders actually need: not synthetic peak numbers, but a practical picture of how a 96 GB Blackwell card handles a very large MoE model under real chat-style loads.

• –Single-user interactive performance looks genuinely strong, with sub-second TTFT on short prompts and around 80 t/s generation.
• –The long-context story is better than expected for a 122B-class model, with only modest token-generation loss even at 65K depth.
• –Concurrency is the real constraint: four short requests scale well enough for batch work, but deep-context chat collapses into multi-second-to-half-minute waits.
• –For teams sizing on-prem inference boxes, this suggests one RTX PRO 6000 can comfortably host premium single-user or light multi-user open-weight chat, but not dense long-context shared serving.
• –The benchmark also reinforces llama.cpp’s growing role as a serious production-ish local serving stack for open-weight models, not just a hobbyist runtime.