Local LLMs hit compression limits

The model charts are improving faster than the surrounding experience, so the bottlenecks are shifting from raw benchmark wins to trust, memory, and sustained usability.

• –Multi-step reasoning still drops off quickly as size shrinks, especially on debugging and planning tasks that need intermediate state preserved.
• –Long context is still ahead of usable context; advertised token windows are rising faster than coherent retrieval and recall.
• –Smaller models can sound fluent while still missing structured outputs, world knowledge, and specific facts, so RAG/search remains the escape hatch for many workflows.
• –MoE is not a free lunch for local users: it often improves speed while pushing the real cost onto VRAM and memory bandwidth.
• –Quantization keeps local models runnable, but recent long-context work suggests 8-bit is mostly safe while 4-bit can bite hard on long inputs; heat and power just make that trade-off more annoying to live with.