OCuLink eGPU crowns mini PC AI homelabs

OCuLink is the "secret sauce" for high-performance edge AI, offering a direct 64Gbps pipe that effectively eliminates bandwidth bottlenecks for single-GPU inference.

• –RX 7800 XT is the clear winner over the 7600 XT for AI; despite having the same 16GB VRAM, its 624 GB/s bandwidth provides roughly 2x the token generation speed for the same model size.
• –PCIe 4.0 x4 bandwidth is nearly indistinguishable from x16 for LLM inference once the model is loaded, as the computation remains memory-bound on the GPU itself.
• –Linux ROCm support allows for surgical GPU isolation using the ROCR_VISIBLE_DEVICES environment variable, enabling a discrete GPU to handle Ollama while the iGPU manages NVR tasks like Frigate.
• –The open-frame G-Dock design with integrated 800W PSU provides superior thermals for 24/7 inference compared to cramped mini PC chassis or enclosed eGPU boxes.
• –OCuLink's lack of hot-plug support is a minor trade-off for the increased stability and reduced system overhead it offers over the USB4/Thunderbolt stack.