OPEN LINK ↗
// 75d agoINFRASTRUCTURE
LocalLLaMA thread diagnoses slow Qwen3.5-27B Q8 on Ollama
A LocalLLaMA user reports very slow generation (about 3.6 tokens/sec) running qwen3.5-27b-Q8_0 through Ollama on an RTX 5090 with 256GB RAM. Replies point to VRAM pressure and context spillover into system RAM as likely bottlenecks, with suggestions to reduce context and use lighter quantizations.
// ANALYSIS
This looks less like a raw hardware problem and more like a local inference configuration mismatch where Q8 plus context can silently trigger costly offloading.
- –Commenters note Q8 for a 27B model can consume most available VRAM, leaving too little room for KV cache.
- –Several responses recommend moving to UD-Q6 or Q4 variants for a better speed/quality tradeoff.
- –The thread highlights a recurring local-LLM issue: convenient defaults in Ollama can obscure performance-critical settings.
// TAGS
qwen3.5-27bollamallminferencegpu
DISCOVERED
75d ago
2026-03-14
PUBLISHED
76d ago
2026-03-13
RELEVANCE
8/ 10
AUTHOR
giveen
