DeepSeek R1 experts draw scrutiny

This is a small Reddit question, but it points at a real systems issue: MoE models are only cheap and scalable if routing stays balanced enough for hardware, while still letting experts specialize.

• –DeepSeek-R1 uses a 671B-parameter MoE architecture with roughly 37B active parameters per token, making expert routing central to both performance and serving cost
• –Research on DeepSeek-R1 expert activations has found localized behavior effects, including refusal-related experts and semantic routing patterns
• –Uniform activation would be convenient for inference, but meaningful specialization almost guarantees some distribution skew across prompts, layers, and domains
• –The useful next benchmark is not just "which experts are hot," but whether hot experts correlate with topic, language, safety behavior, or reasoning mode