LocalLLaMA thread explains model hardware needs

This is the standard local-LLM onboarding question, and the answer is less about "can it fit?" than "how fast can you serve tokens?" That distinction matters because a model that technically runs on a CPU may still feel unusable if bandwidth is the bottleneck.

• –Parameter count mostly determines weight memory, so scaling from 1B to 7B to 13B increases RAM/VRAM needs roughly linearly
• –Quantization is the main lever for shrinking models enough to run on consumer hardware without full precision overhead
• –A 1B model can run on CPU with tools like `llama.cpp`, but inference will be much slower than on a GPU
• –Dense models stress memory bandwidth more than mixture-of-experts models, which can make MoE models more forgiving on constrained hardware
• –For practical local use, the real decision is usually whether your system has enough fast memory and bandwidth, not whether it has a GPU at all