OPEN LINK ↗
// 45d agoINFRASTRUCTURE
llama.cpp users test mixed GPUs
A LocalLLaMA user asks whether a 16GB RTX 4070 Ti Super and a 12GB RTX 2080-class card can be combined for llama.cpp inference across Windows, Ubuntu VM, and Proxmox. The short answer is yes in principle, but uneven VRAM, older CUDA support, and cross-machine latency make the setup more useful for experimentation than clean speed scaling.
// ANALYSIS
Mixed-GPU local inference is workable, but it is not the same as magically pooling VRAM into one fast card.
- –llama.cpp can split model layers across multiple GPUs, and uneven cards usually need explicit weighting with options such as tensor split rather than a simple 1:1 setup
- –Different NVIDIA generations can coexist, but the oldest card tends to constrain driver and CUDA choices
- –Splitting across separate machines or VMs pushes users toward llama.cpp RPC, where network latency can erase much of the benefit unless the link is fast
- –The practical win is fitting larger quantized GGUF models; throughput may still bottleneck on the slower GPU or PCIe/network path
// TAGS
llama-cppllminferencegpuself-hostedopen-source
DISCOVERED
45d ago
2026-04-23
PUBLISHED
45d ago
2026-04-22
RELEVANCE
7/ 10
AUTHOR
smolpotat0_x