REDDIT · REDDIT// 29d agoINFRASTRUCTURE

LocalLLaMA thread diagnoses slow Qwen3.5-27B Q8 on Ollama

A LocalLLaMA user reports very slow generation (about 3.6 tokens/sec) running qwen3.5-27b-Q8_0 through Ollama on an RTX 5090 with 256GB RAM. Replies point to VRAM pressure and context spillover into system RAM as likely bottlenecks, with suggestions to reduce context and use lighter quantizations.

// ANALYSIS

This looks less like a raw hardware problem and more like a local inference configuration mismatch where Q8 plus context can silently trigger costly offloading.

–Commenters note Q8 for a 27B model can consume most available VRAM, leaving too little room for KV cache.
–Several responses recommend moving to UD-Q6 or Q4 variants for a better speed/quality tradeoff.
–The thread highlights a recurring local-LLM issue: convenient defaults in Ollama can obscure performance-critical settings.

// TAGS

qwen3.5-27bollamallminferencegpu

DISCOVERED

29d ago

2026-03-14

PUBLISHED

29d ago

2026-03-13

RELEVANCE

8/ 10

AUTHOR

giveen