BACK_TO_FEEDAICRIER_2
Routerly 0.2.0 ships semantic routing
OPEN_SOURCE ↗
REDDIT · REDDIT// 3d agoBENCHMARK RESULT

Routerly 0.2.0 ships semantic routing

Routerly’s next release is being shaped by a benchmark campaign that compared semantic-intent routing with direct Claude Sonnet 4.6 on MMLU, HumanEval, and BIRD. The big changes: an embedding-based router becomes the default, routing cost is now exposed per request, and the SQL pool is being rebuilt after the BIRD result exposed a backend failure.

// ANALYSIS

The smart move here is not pretending routing “wins” everywhere, but using the data to tighten the defaults and make the tradeoffs visible. That is how an LLM gateway earns trust: measure the overhead, separate infra failures from policy failures, and let the cheaper policy become default when it holds up.

  • Semantic-intent routing looks like the right default because it matched or beat the LLM router while being dramatically cheaper to run.
  • Surfacing routing cost versus inference cost is the kind of observability that prevents routing from becoming a black box with hidden tax.
  • The BIRD miss is a reminder that bad backend pools can distort evals and get blamed on the router if the system isn’t instrumented well enough.
  • Raising the campaign to 55 seeds is the credible move; small-sample benchmark claims are too noisy for modest gaps.
  • Routerly is still not claiming it can beat Opus-class models on the hardest slices, which makes the product argument more honest: route most traffic cheaply, reserve premium models for the cases that justify them.
// TAGS
routerlyllmembeddingbenchmarkself-hostedinference

DISCOVERED

3d ago

2026-04-08

PUBLISHED

4d ago

2026-04-08

RELEVANCE

9/ 10

AUTHOR

nurge86