Inworld slashes Gemma 4 26B latency
Inworld has rolled out an optimized Gemma 4 26B serving pipeline delivering a 0.23-second p95 time to first token, up to 8.6× faster than the median provider on OpenRouter. The build lowers input pricing by nearly 40% to $0.07 per million tokens with context caching at $0.035, targeting latency-sensitive consumer and voice applications.
Commodity inference providers compete on margins, but real-time consumer AI demands radical TTFT optimization that standard multi-tenant setups fail to deliver. By driving 26B-class latency down to 230ms, Inworld makes open models practical for voice agents and interactive characters where multi-second lags break immersion.
- –Sub-250ms p95 time to first token makes 26B models competitive with lightweight sub-8B models for real-time turn taking
- –Aggressive pricing at $0.07/M input and $0.34/M output undercuts general-purpose model aggregators
- –Context caching at $0.035/M tokens significantly reduces continuous dialogue costs for long-running sessions
- –Drop-in OpenAI-compatible API allows developers to bypass self-hosted GPU infrastructure without switching client libraries
DISCOVERED
44m ago
2026-11-19
PUBLISHED
2h ago
2026-11-19
RELEVANCE
AUTHOR
inworld