Ranvier open-sources prefix-aware LLM router

This is a very real infrastructure idea, not a gimmick: once KV cache locality matters, round-robin routing starts wasting expensive GPU work. Ranvier sits in the same emerging lane as vLLM Router and llm-d, but its portability-first pitch could make it easier to adopt.

• –The biggest win is avoiding redundant prefill on repeated prefixes, which is exactly where RAG, multi-turn chat, and few-shot workloads bleed latency.
• –The tradeoff is that Ranvier infers cache state from routing history instead of reading backend internals, so it gives up some precision for broad backend compatibility.
• –The gains appear strongest on larger models and shared-context traffic; smaller models are less likely to feel the P99 benefit as sharply.
• –It only helps if prefix caching is actually enabled in the serving backend, so this is a routing layer, not magic cache creation.
• –Apache 2.0 licensing makes it easier for teams to test in existing stacks without committing to a new serving engine.