Unity Android LLM hits 16.6 tok/s

This is a deployment win more than a model win: the key lesson is that on mobile GPUs, the fastest quantization is not always the smallest one.

• –The speedup is driven by backend choice, not a new model architecture, which makes this useful for anyone trying to ship edge inference in real apps
• –The Q8_0 vs Q4_0 result is the important takeaway; dequantization overhead can erase the theoretical gains of lower-bit quantization on Adreno
• –The Unity bridge detail matters because native LLM structs are brittle across P/Invoke boundaries; the C wrapper is the difference between a demo and a shippable integration
• –The benchmarks also show how bad the alternatives were in practice: CPU inference was unusable, QNN barely hit the NPU, and Unity’s renderer broke the GPU path for LiteRT-LM
• –This is strong evidence that Android game and app teams should treat llama.cpp + OpenCL as a serious fallback path for offline generation on Snapdragon devices