Local LLM TPS floor depends on interactivity

iGPU-based local LLM setups are transitioning from curiosities to functional async tools, but the user experience floor is still governed by human reading speeds.

• –5–10 TPS is the "interactive floor" for real-time chat, while batch processing remains viable at speeds as low as 1–2 TPS.
• –"Thinking" models like O1-style are shifting the performance metric from raw output speed to the quality of the reasoning process.
• –Intel iGPUs (12900HK) can surprisingly support 30B parameter models like Qwen 3.5-A3B, challenging the assumption that dGPUs are mandatory.
• –Software fragmentation remains a hurdle; OpenVINO support in llama.cpp is described as a "nightmare" compared to Vulkan or Sycl runtimes.