Qwen, Kimi, GLM test 5090 limits

Quantization helps a lot, but it does not change the basic math: once you move into 300B dense territory, a single 64GB GPU runs out of headroom fast. The real nuance is that MoE models can look enormous on paper while still having a much smaller active-parameter footprint at inference.

• –Qwen’s official repo shows 72B int4 using about 48.9GB, which fits on 64GB only with limited room left for context, KV cache, and runtime overhead
• –60B-class dense models are the sensible upper tier for this setup if you want decent speed and fewer OOM headaches
• –300B dense models would need roughly 150GB just for 4-bit weights before cache and allocator overhead, so they are not realistic on one consumer GPU
• –MoE models like Kimi K2 are easier to misread: 1T total parameters sounds impossible, but 32B active parameters makes the runtime story much closer to a large 30B-class model
• –The bottleneck after weights is context length, so long prompts and long chats can eat the extra memory you thought quantization bought you