DeepMind Proposes New Model Routing Framework

Evaluating routers purely on accuracy is a dangerous illusion that hides redundant or highly erratic model dispatching.

* Current benchmarks miss the point: Standard benchmarks evaluate routers on overall accuracy, ignoring that a router might look good but do nothing useful if the model pool is redundant.

* Diminishing returns on model variety: The discovery that under ten models are sufficient to capture the vast majority of diversity (HSE) implies that developer teams should focus on a tiny, highly differentiated coreset of models rather than maintaining large, complex model societies.

* KNN vs. Prompted routing trade-off: The collapse of KNN routers under minor prompt perturbations indicates that learning-based dispatchers are highly brittle, making prompted routing a far more reliable choice for real-world deployments.