Llama.cpp users debate 128GB VRAM gains

The interesting takeaway is that extra VRAM looks more useful for workflow design than for unlocking a dramatically better coding model tier.

• –Several commenters argue 96GB already covers the sweet spot for local 80B-120B class models at practical quants, so 128GB does not suddenly create a huge new frontier for coding quality
• –The strongest use case for the second GPU is running parallel capabilities like Qwen3-Coder-Next, STT, TTS, or image generation instead of forcing a single giant model to span a slow interconnect
• –Bandwidth, not raw memory, is the bottleneck once a model crosses one card boundary, especially without NVLink and with Thunderbolt in the path
• –The thread also surfaces a practical llama.cpp issue: the poster reports random-token failures with `-sm layer` on Qwen 3.5 that disappear with `-sm row`, which matters for anyone experimenting with multi-GPU sharding
• –For AI developers, this is less a “buy more VRAM for one better model” story and more a case for building a resident local toolchain with coding, orchestration, and media models always ready