OPEN LINK ↗
// 66d agoNEWS
Local LLM TPS floor depends on interactivity
A LocalLLaMA community discussion establishes 5–10 tokens per second (TPS) as the minimum for real-time chat, while asynchronous tasks remain functional at much lower speeds. The conversation highlights the growing viability of iGPUs for local inference of 30B parameter models.
// ANALYSIS
iGPU-based local LLM setups are transitioning from curiosities to functional async tools, but the user experience floor is still governed by human reading speeds.
- –5–10 TPS is the "interactive floor" for real-time chat, while batch processing remains viable at speeds as low as 1–2 TPS.
- –"Thinking" models like O1-style are shifting the performance metric from raw output speed to the quality of the reasoning process.
- –Intel iGPUs (12900HK) can surprisingly support 30B parameter models like Qwen 3.5-A3B, challenging the assumption that dGPUs are mandatory.
- –Software fragmentation remains a hurdle; OpenVINO support in llama.cpp is described as a "nightmare" compared to Vulkan or Sycl runtimes.
// TAGS
llama-cppllminferencegpuopen-source
DISCOVERED
66d ago
2026-03-23
PUBLISHED
66d ago
2026-03-23
RELEVANCE
7/ 10
AUTHOR
ShaneBowen