LiteLLM, llama.cpp tackle role-based routing

This is a sensible pattern only if you are truly VRAM-bound; otherwise model choreography can add more latency and operational noise than it saves.

• –LiteLLM covers the routing layer well, but it does not solve container or model lifecycle by itself
• –llama.cpp router and Ollama handle load/unload behavior more directly, which matters for local stacks with tight memory budgets
• –Per-agent model selection is already common in agent frameworks, so the missing piece is usually policy and serving infrastructure, not editor plugins
• –Cold starts and state handoff are the real tax; a single stronger model with role-specific prompts or configs may outperform a multi-model setup
• –If you do want specialization, keep roles explicit in config and put one router in front of interchangeable backends