Devs mix RTX 4070, 5070 for LocalLLaMA

Pooling VRAM across mixed GPU generations remains the quintessential hacker approach to local AI, offering a budget-friendly alternative to enterprise hardware.

• –Inference engines like llama.cpp easily split model layers across disparate NVIDIA cards
• –Combining a 12GB and 16GB card yields 28GB total VRAM, unlocking quantized 70B models
• –Generation speed is bottlenecked by the slower card, but the capacity increase outweighs latency tradeoffs
• –Physical spacing, PCIe lane distribution, and power supply limits pose the biggest hurdles for DIY dual-GPU setups