llama.cpp splits LLMs across GPUs

This is a classic local-LLM scaling question: yes, you can shard weights across GPUs, but on cheap older cards the upgrade buys capacity first, speed second.

• –`llama.cpp` supports multi-GPU splitting, and vLLM documents single-node tensor parallelism for models that do not fit on one card.
• –On Pascal-era mining GPUs, PCIe communication can become the bottleneck, so two cards rarely behave like one clean 12GB GPU.
• –Quantization and context size matter as much as raw weights; KV cache can eat the headroom you thought you gained.
• –If the stack is plain Transformers, `device_map="auto"` is not the same thing as true tensor parallelism.
• –For this use case, the smarter tradeoff is often a smaller quantized model before buying a second used GPU.