techNmak releases 35-chapter MoE handbook

MoE is frequently oversimplified as free parameter capacity, but in practice, model efficacy hinges far more on distributed communications and memory bandwidth than raw routing heuristics. The handbook clarifies why activating a fraction of parameters per token does not reduce inference memory footprints since full weights remain in VRAM, creating distinct throughput tradeoffs across batch sizes. It also bridges early mathematical foundations with modern stabilization techniques like DeepSeek-V3's auxiliary-loss-free bias routing, while explaining how token dispatch, all-to-all communication overhead, and dropless execution impact multi-GPU clusters. By pairing concrete mental models with worked numerical examples, it bridges the gap between high-level overviews and academic research for systems engineers.