Google AI Edge ships LiteRT-LM

This is Google turning “local LLMs” from a side project into a real platform layer. The interesting part is not just edge inference, but the full pipeline around it: session state, KV cache, prompt caching, and function calling.

• –Built on LiteRT, so it inherits Google’s edge runtime while adding LLM-specific orchestration on top
• –Cross-platform support lowers the cost of shipping one on-device stack across mobile, desktop, and embedded targets
• –GPU and NPU acceleration make it much more credible than CPU-only local model demos
• –Multimodal inputs and tool use push it toward agentic, app-integrated workflows rather than plain chat
• –The tradeoff is still model footprint and device fragmentation, which will decide whether developers adopt it beyond prototypes