Ryzen AI Max cluster sparks RDMA debate

This is the real frontier for local AI builders right now: not just bigger GPUs, but whether clever interconnects can turn a few prosumer boxes into something cluster-like for tensor parallel inference.

• –The idea is grounded in real community experimentation, with recent Strix Halo posts showing RoCE v2-based distributed inference setups are already being tested in the wild.
• –RoCE v2 offers the same broad promise as RDMA over Thunderbolt—lower latency and direct-memory-style transfers—but it is not a plug-and-play clone of Apple’s stack and depends heavily on NICs, drivers, and software support.
• –In practice, PCIe lane limits, slot width, risers, and motherboard layout can become a bigger constraint than raw link bandwidth, especially when trying to keep an RTX 3090 fully fed.
• –For AI developers, the thread is a useful signal that local inference infrastructure is getting more ambitious, but the systems engineering burden is still high compared with buying a single larger accelerator.