Perplexity Orchestrator API adds KV-cache compatible routing
Perplexity's Orchestrator API now supports intelligent decision-model routing that preserves KV-cache across sessions. This allows developers to maintain prompt caching benefits in multi-turn conversations even when requests are distributed across different infrastructure nodes.
This update addresses one of the biggest hurdles in using high-performance model routing in production — maintaining stateful cache efficiency without manual intervention.
- –Intelligent routing automatically handles session stickiness, ensuring conversation context isn't lost or forced to re-compute on new nodes.
- –Developers get automatic cost savings through cache-read pricing without having to manage complex cache anchors or routing hints.
- –This elevates the Orchestrator API from a simple load balancer to a sophisticated platform-aware infrastructure layer.
- –It significantly simplifies the architecture for high-concurrency LLM apps, letting developers focus on model choice rather than infrastructure state.
DISCOVERED
1h ago
2026-11-17
PUBLISHED
1h ago
2026-11-17
RELEVANCE
AUTHOR
AravSrinivas