LocalLLaMA thread weighs 5090, RTX Pro 6000

The GPU pair is impressive, but for coding assistants the real bottleneck is usually model quality, context handling, and serving efficiency, not just raw VRAM. This is a classic local-LLM reality check: bigger hardware expands the menu, but it does not automatically beat the best smaller code models.

• –A 32GB 5090 is already enough for fast dense 20B-30B coding models with decent headroom for context and tool use
• –The RTX Pro 6000 mainly buys flexibility for 70B+ or 128B-class runs, not a guarantee of better coding output
• –Offloading to system RAM is a fallback, but it typically hurts latency enough to undermine the “replacement for paid models” goal
• –PCIe bottlenecks matter less for inference than many people expect; serving stack, batching, and prompt length often dominate user experience
• –The best test is real coding tasks, not token-per-second bragging rights, because agent quality and long-context reliability decide whether the setup is actually useful