Qwen2.5 1.5B disappoints, 7B crawls

This is a very normal local-inference bottleneck, not a bad model problem. Qwen2.5 itself spans sizes from 0.5B up to 72B, so the real constraint here is the 4GB GPU, not model availability.

• –1.5B is in the “fast enough to run, not smart enough to trust” zone for many general-purpose tasks
• –7B is the first size that starts feeling meaningfully better, but on a 1050 Ti it usually means heavy CPU offload or aggressive quantization, which tanks latency
• –A 3B-class model is often the more practical middle ground on older consumer hardware
• –Tightening context length, using a faster runtime, and keeping expectations focused on narrow tasks will matter more than chasing a bigger model
• –The post is useful as a hardware reality check for anyone trying to self-host an LLM on aging desktop parts