OPEN LINK ↗
// 53d agoOPENSOURCE RELEASE
Triton MoE kernel beats CUDA at inference
A pure Triton implementation of Mixture-of-Experts dispatch outperforms Stanford's CUDA-optimized Megablocks for inference batch sizes. By fusing gate and up projections, the kernel reduces memory traffic by 35% and maintains high performance across NVIDIA and AMD hardware.
// ANALYSIS
This project demonstrates that Triton is no longer just a "fast enough" alternative to CUDA; it can actually exceed hand-tuned C++ when algorithmic fusion is prioritized.
- –Fusing gate+up projections into a single tile load eliminates nearly 500MB of intermediate buffers, a massive win for memory-bound MoE inference.
- –The block-scheduled grouped GEMM handles variable expert batch sizes without the padding overhead that typically plagues naive MoE implementations.
- –Achieving 131% of Megablocks' speed at small batch sizes (32 tokens) makes this specifically relevant for real-time chat and agentic workflows.
- –Zero-change portability to AMD MI300X highlights the growing maturity of the Triton ecosystem for multi-vendor AI infrastructure.
// TAGS
triton-moe-dispatchtritonllminferencegpumixtraldeepseekopen-source
DISCOVERED
53d ago
2026-04-05
PUBLISHED
53d ago
2026-04-05
RELEVANCE
8/ 10
AUTHOR
bassrehab
