Prime Mega MoE Accelerates Qwen3 Training

// 1h agoBENCHMARK RESULT

Prime Mega MoE Accelerates Qwen3 Training

Prime Intellect extended DeepSeek’s Mega MoE kernel with an overlapping backward pass, making it trainable and delivering up to 55% faster MoE training than TorchTitan and 34% faster hosted training with prime-rl. [Official benchmark](https://www.primeintellect.one/blog/prime-mega-moe)

// ANALYSIS

This targets MoE’s real bottleneck—communication overhead—rather than chasing another model release. The gains look significant, but depend heavily on H200/Blackwell hardware and carefully tuned distributed-training configurations.

–Fuses dispatch, expert computation, and combine operations while overlapping communication with GPU compute.
–Benchmarked on Qwen3-30B-A3B and Qwen3-235B-A22B at 16K sequence lengths with expert parallelism and FP8.
–The backward kernel is the key upgrade over DeepSeek’s forward-only implementation.
–The results strengthen prime-rl’s case as a high-performance stack for large-scale MoE post-training.
–Hopper and Blackwell support limits portability across older or non-NVIDIA systems.

// TAGS

prime-mega-moemoetrainingtraining-infragpubenchmark

DISCOVERED

1h ago

2026-10-29

PUBLISHED

1h ago

2026-10-29

RELEVANCE

9/ 10

AUTHOR

PrimeIntellect

// KEEP READING

More AI developer news from the feed

EXPLORE FULL FEED

TUTORIAL48m ago

tiny-llm Turns vLLM Internals Into Coursework

tiny-llm is an open-source, four-week course that teaches LLM inference by rebuilding Qwen3 serving components on Apple Silicon with MLX. Students implement operators, kernels, KV caching, continuous batching, paged attention, and a basic coding agent.

LAUNCH52m ago

Unity Plugin Joins Antigravity Marketplace

Unity’s official agent plugin is now available in Google Antigravity’s Plugin Marketplace, enabling one-click access to Unity-aware skills, CLI control, and MCP-based Editor workflows. It supports Unity 6+ and helps agents build, simulate, and manage real-time 3D projects.

RESEARCH1h ago

The Inbox Reframes Agent Coordination

MIT Media Lab researchers argue that large agent societies need recipient-controlled inboxes, not just push-based message delivery. Simulations of up to 16,000 agents show that pull-based reading and memory rules can preserve coordination under heavy traffic. [arXiv](https://arxiv.org/abs/2610.36264)