Qwen3-Coder 30B hits hardware wall

This is the classic "frontier model, budget box" mismatch. The user’s goals are sensible - offline use, privacy, and fine-tuning - but the hardware is the limiting factor, not the choice of quantization.

• –4GB VRAM is the main bottleneck; even heavily quantized 70B-100B models will be slow and memory-starved on this setup.
• –MoE helps efficiency, but it does not magically make huge reasoning models comfortable on consumer-grade hardware.
• –Smaller open-weight models in the 7B-14B range, or maybe a carefully quantized ~27B model, are the realistic sweet spot for speed and usability.
• –GLM-5 and Kimi K2.5 are better viewed as API-first reasoning models than something you should expect to run well on this machine.
• –If the goal is serious local work, a GPU upgrade or multi-GPU server matters more than chasing one giant model.