7900 XTX Multi-GPU Support Still Bumpy

The short version: yes, tensor parallelism exists now, but the software stack is still the limiting factor, not the idea of splitting a model across two 7900 XTX cards.

• –vLLM’s current docs list Radeon RX 7900 series support on ROCm 6.3+ and document tensor parallelism as a normal multi-GPU mode
• –AMD’s ROCm docs still warn that two 7900 XTXs need proper CPU-controlled PCIe slots; chipset-attached cards can trigger HIP errors in AI workloads
• –A March 2026 ROCm issue reports dual-GPU collectives failing on 2x 7900 XTX with ROCm 7.2.1/RCCL 2.27.7, and the reporter hit it while trying to run vLLM tensor parallelism
• –vLLM has continued to improve ROCm support, but the presence of open 7900 XTX-specific bugs means consumer RDNA3 is still more "hands-on lab setup" than "drop-in production box"
• –llama.cpp also now marks parallel multi-GPU support on ROCm, which is a real improvement, but it does not remove topology and collective-communication caveats