MiMo Distill 9B Adds GPU-Friendly Quants

This makes an unusual local model substantially easier to run, but the convenience comes with meaningful evaluation and safety caveats.

• –Q4_K_M brings the model into roughly 6GB territory, making 12GB GPUs much more practical targets.
• –NVFP4 W4A8 and W4A4 offer alternative memory and throughput trade-offs for supported hardware.
• –The base model’s ablation removes refusal behavior, but its own audit found most removed refusals became soft deflections rather than substantive answers.
• –Quantization can further affect capability, so developers should benchmark coding, tool use, and safety behavior on their own workloads before deployment.
• –This is best viewed as an enthusiast-friendly local inference experiment, not a production-ready aligned assistant.