DeepSeek Flash-1017 Hits 500 Tok/s Locally

This is an impressive inference-engineering result, but the headline number is highly dependent on workload, context, batching, and kernel implementation.

• –SSD n-gram offload makes the 552B-parameter MoE model practical without keeping every auxiliary component in GPU memory.
• –A megakernel can significantly reduce launch and synchronization overhead across the decode path.
• –The result highlights how DeepSeek’s sparse architecture and compressed KV-cache design favor unconventional local-serving systems.
• –Four workstation GPUs and high-speed storage remain a serious homelab investment, so this is not yet “local” in the consumer sense.
• –Reproducible benchmarks should report prompt length, output length, quantization, concurrency, and sustained versus burst throughput.