DeepGEMM adds Blackwell support, Mega MoE kernels

DeepSeek is aggressively front-running the industry by building Blackwell-native infrastructure before the hardware is even widely available.

• –Mega MoE fuses dispatch and combine operations into a single kernel, ruthlessly optimizing NVLink-heavy distributed training.
• –FP4 support for weights and MQA logits suggests DeepSeek's next model will be massive enough to require 4-bit precision to fit on standard clusters.
• –Early Blackwell (SM100) support includes fifth-gen Tensor Core optimizations, placing DeepSeek at the bleeding edge of NVIDIA's new architecture.
• –The library requires PyTorch 2.9+, indicating a reliance on the very latest compiler features for JIT kernel execution.
• –This release confirms that "Expert Parallelism" efficiency is the primary architectural bottleneck DeepSeek is currently solving.