Qwen3.5-27B crawls on 2080 Ti

This is a useful real-world reminder that dense 27B models get ugly fast when your local setup barely fits them. For AI developers running open-weight models at home, memory behavior matters as much as raw parameter count.

• –The headline number is the ~3.5 t/s decode speed, which makes an otherwise strong open-weight model feel impractical on older single-GPU rigs
• –Commenters point to RAM spillover and incomplete GPU offload as the likely culprit, not an inherent flaw in Qwen3.5-27B
• –Several replies suggest newer `llama.cpp` builds, lower context, and full-VRAM loading can materially improve throughput, with the OP reporting a bump to 5 t/s after tweaks
• –The thread also highlights a broader local-inference lesson: dense models punish “just barely enough VRAM” setups much harder than smaller or sparse alternatives
• –For developers choosing local stacks, this kind of community benchmark is more actionable than vendor specs because it exposes the real tuning pain on aging consumer hardware