Gemma 4 quant debate heats up

This is less about one universally “best” quant and more about how much speed, quality, and runtime stability you can squeeze out of your hardware. The emerging pattern is clear: 26B A4B is the sweet spot for most local users, while 31B is the quality-first pick if you can afford the memory.

• –Google’s own release frames 26B A4B as the balanced option and 31B as the strongest model, which matches the community split in the thread.
• –Bartowski’s Q4_K_M is getting praise for throughput and day-to-day usability, especially for long-context and coding-heavy sessions.
• –Unsloth’s Gemma 4 support is solid and well-documented, but several community posts suggest llama.cpp versioning and quant choice still materially affect real-world behavior.
• –The discussion reinforces a familiar local-LLM rule: the “best” quant is usually the one that fits your VRAM, your runtime, and your workload, not just the biggest file.