MoE Models Lack Cheap Adapter Paths

The core issue is not just training cost; it is that MoE architectures make routing part of the problem, so a generic “LoRA marketplace” does not port cleanly across models the way it does for diffusion checkpoints.

• –Recent work already points to workable paths: expert-specialized fine-tuning, MoE-LoRA variants, router-guided adapters, and inference-time expert composition.
• –The hard part is coordination: if routing is off, you get expert collapse, wasted capacity, or only a few experts learning while the rest stay cold.
• –Even “cheap” methods still need custom infra for loading, switching, batching, and evaluating experts, which raises the bar for hobbyists and small teams.
• –Dense models dominate adapter culture because the tooling is simpler, the behavior is easier to predict, and the adapter artifacts are more reusable across releases.
• –Net: MoEs are not adapter-proof, but they are adapter-fragmented, and that fragmentation is why the ecosystem is thin.