Dual GPUs vs Bigger Card for 31B LLMs

Hot take: for interactive local inference, more VRAM on one card is usually cleaner and more predictable than splitting across two consumer GPUs unless the model simply will not fit.

• –`llama.cpp` supports multi-GPU splitting, but that does not remove synchronization overhead; it mainly helps fit larger models.
• –`vLLM` has first-class tensor-parallel inference, so it is the more natural choice if the goal is multi-GPU serving rather than single-user tinkering.
• –A 32 GB card is a meaningful step up because it reduces or eliminates the need to shard weights and leaves more room for KV cache and longer contexts.
• –Dual 16 GB cards can be a practical capacity play, but they usually trade simplicity and latency consistency for extra tuning and bus overhead.
• –For 27B to 31B models, the real decision is often “fit comfortably on one device” versus “fit at all across devices,” not “scale linearly with more GPUs.”