Gemma 4 31B bogs down local GPUs

You probably are not missing a secret setting. This looks like a hardware-fit problem, not a broken model: Gemma 4 31B is the top-end dense variant, and on a 16GB GPU with only 32GB system RAM, it can end up paging and dragging.

• –Google positions Gemma 4 31B as the large dense model in the family, while the smaller E2B/E4B and 26B MoE options are the ones meant to feel more local-friendly
• –The thread’s key warning is KV cache overhead: the model itself is around 20 GB, and context growth can eat the remaining RAM fast
• –If you are running Q4 quantization in LM Studio or Ollama, the model can still oversubscribe VRAM and fall back to slower CPU/system-memory paths
• –For this class of hardware, shorter context, a smaller quant, or Gemma 4 26B MoE is more likely to feel responsive than 31B dense
• –The symptom pattern here is classic first-token latency plus low tok/s, which usually means memory bottlenecks rather than thermal throttling or a bad prompt