Llama.cpp enables MXFP4 on older NVIDIA GPUs

Software-level emulation of Blackwell-exclusive features democratizes high-efficiency inference for the massive installed base of older GPUs.

• –MXFP4 provides a superior perplexity-to-size ratio compared to standard 4-bit GGUF quants, making it a "sweet spot" for mid-sized models.
• –On Ampere and Ada cards, performance relies on software dequantization (DP4A) rather than hardware acceleration, trading some inference speed for significantly better model quality.
• –The Qwen 3.6 35B A3B model's sparse architecture (activating only 3B parameters) makes it uniquely viable for 10GB-12GB VRAM cards when paired with this format.
• –While native Blackwell support yields a 25-33% speedup, the open-source community's ability to backport these formats ensures hardware longevity for hobbyists.
• –Developers should still benchmark against IQ4_XS, as the software overhead of MXFP4 on older cards can vary significantly depending on the specific kernel implementation.