KALAVAI predicts when specialist fusion works

This is a genuinely interesting result because it turns model merging into a measurable planning problem instead of a hope-and-pray ensemble trick. The strongest claim isn’t just that fusion works, but that teams can estimate fusibility before spending compute.

• –The best gains show up where specialists are truly complementary, especially cross-lingual and private-domain setups where the base model is weak.
• –The divergence rule is promising, but it is still a small-sample heuristic: the line is fit on six conditions, so broader replication matters.
• –The protocol is refreshingly simple operationally: shared initialization, independent fine-tunes, no gradient sharing, and a 500-step linear router on standard PyTorch and Hugging Face.
• –The latency bill is the obvious tradeoff: every specialist runs at inference time, so this favors quality, privacy, or data isolation over throughput.
• –The comparison against equal-compute monolithic training is the right sanity check, and it suggests cooperative specialization is doing something a single mixed model does not.