llama.cpp, ik_llama.cpp land native FP4 support

// 98d agoOPENSOURCE RELEASE

llama.cpp, ik_llama.cpp land native FP4 support

Llama.cpp and the ik_llama.cpp fork have integrated native FP4 support, introducing the GGML_TYPE_NVFP4 and GGML_TYPE_MXFP4 formats. This milestone enables high-precision 4-bit inference with significant VRAM savings and hardware-level acceleration on NVIDIA's Blackwell GPUs, bridging the gap between quantization efficiency and model performance.

// ANALYSIS

Native FP4 is a paradigm shift that brings hardware-level efficiency to local LLM quantization without the quality trade-offs of integer methods.

–NVFP4 enables native Blackwell Tensor Core acceleration for up to 2.3x inference speedups.
–MXFP4 provides a cross-platform 4-bit float standard for CPU and non-Blackwell GPU backends.
–Direct collaboration with NVIDIA ensures seamless support for next-gen models like gpt-oss.
–The implementation significantly reduces the VRAM barrier to running large models on consumer-grade hardware.
–Evolution of the GGUF format to support block-scaled FP4 weights preserves model fidelity during quantization.

// TAGS

llama-cppllminferencegpuopen-source

DISCOVERED

98d ago

2026-04-25

PUBLISHED

98d ago

2026-04-25

RELEVANCE

9/ 10

AUTHOR

Usual-Carrot6352

// KEEP READING

More AI developer news from the feed

EXPLORE FULL FEED

TUTORIAL30m ago

Dual Blackwell GPUs run 167 GB DeepSeek-V4 FP8

A developer shared a deployment recipe for running the official FP8 version of DeepSeek-V4-Flash-0731 alongside DSpark speculative decoding on a dual NVIDIA RTX PRO 6000 Blackwell (SM120) GPU rig. Requiring approximately 167 GB of VRAM, the model fits cleanly across the system's combined 192 GB VRAM capacity (2× 96 GB) without offloading or truncation.

UPDATE1h ago

Genspark Workspace 6.0 drops six major updates

Genspark Workspace 6.0 expands Genspark's ecosystem across six core updates designed to bridge ambient work context into executable workflows. Key releases include SecondBrain Note hardware voice recorder, GenTeam multi-agent collaboration, GenMail email workflows, Genspark Design, AI Slides, and AgentBase for custom databases.

NEWS1h ago

Google begins active development on Gemini 4

Google is reportedly actively developing Gemini 4, its next-generation foundation model designed to be its most advanced AI system to date. Key objectives for the new model include superior reasoning skills, improved coding assistance, and enhanced agentic capabilities for autonomous task execution, while Gemini 3.5 Pro continues testing behind the scenes.