Wayfinder Router routes LLM queries offline
Wayfinder Router is an open-source CLI tool and library that routes prompts deterministically between local and cloud-based LLMs by analyzing their structural complexity offline. Running in microseconds, it acts as an OpenAI-compatible proxy, directing simple queries to local models and complex ones to the cloud to reduce API costs without adding network latency.
While semantic LLM routers offer higher accuracy, they introduce latency and overhead that negate the savings of local execution; Wayfinder's structural approach is a practical, cost-effective compromise for developers who want offline, zero-latency routing.
* Predictable and offline-first design eliminates routing cost and latency entirely.
* Calibration feature allows developers to fine-tune score weights to match their specific application traffic.
* Out-of-the-box support for OpenAI-compatible endpoints makes integration seamless with zero client code modifications.
* Lexical cue routing is off by default, acknowledging that custom keyword heuristics often fail to generalize.
* Simple prompts (like typos or basic summaries) are successfully kept local, reducing frontier API bills.
DISCOVERED
1h ago
2026-06-28
PUBLISHED
5h ago
2026-06-28
RELEVANCE
AUTHOR
handfuloflight