Local server build eyes Claude Code replacement

This is a community question, not a product announcement — but it surfaces a real and growing trend of developers self-hosting capable coding models to escape API costs and privacy concerns.

• –96GB VRAM across 4x RTX 3090s can comfortably run 70B parameter models (e.g., Llama 3.3 70B, Qwen2.5 72B) in Q4/Q5 quantization
• –NVLink pairs help with tensor parallelism within each pair, but cross-pair communication over PCIe x4 is a bottleneck for large unified inference
• –Multi-model setups (e.g., a fast small model for autocomplete + a large model for complex reasoning) can outperform a single huge model for coding workflows
• –Ollama, vLLM, and llama.cpp are the standard serving stacks for this hardware class
• –DeepSeek Coder V2 and Qwen2.5-Coder 32B are community favorites for coding at this VRAM range