DeepSeek Open-Sources DeepInfer Inference Engine

DeepInfer is a meaningful look at DeepSeek’s production optimizations, but it is not yet a turnkey replacement for vLLM or SGLang.

• –Supports NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs, widening hardware options for self-hosted deployments
• –Includes FlashMLA, DeepGEMM, DeepEP, DSpark verification, and expert-parallel execution
• –Requires developers to supply key service components, including KV caching, prefill/decode disaggregation, and replica load balancing
• –Limited initially to DeepSeek-V4.1-Flash, making it more valuable as a specialized reference implementation than a general-purpose engine
• –Built primarily on vLLM, with ideas from SGLang and TensorRT-LLM, giving the broader inference ecosystem new optimizations to study