llama.cpp CPU thread tradeoffs resurface

More cores help only until the memory subsystem stops feeding them; for CPU-only LLM inference, old cheap server silicon can look attractive on capacity but disappoint on tokens per second.

• –llama.cpp exposes thread controls and recommends tuning around physical cores, not blindly maxing logical threads.
• –DDR3 is the real warning sign: large quantized models stream weights constantly, so memory bandwidth can dominate generation speed.
• –Dual-socket Xeons add RAM capacity and memory channels, but NUMA penalties can make “all cores” slower than a carefully pinned subset.
• –A faster modern CPU with AVX2/AVX-512, stronger single-core performance, and DDR4/DDR5 may beat an older high-core-count box for interactive use.
• –If the goal is hosting huge models cheaply and patiently, prioritize RAM capacity, memory channels, and measured llama.cpp benchmarks over core count alone.