Strix Halo cluster weighs llama.cpp RPC

Distributed inference on Strix Halo looks like a capacity play, not a free throughput win.

• –llama.cpp’s RPC backend splits model weights and KV cache across local and remote devices by available memory, so the host can still participate in inference rather than acting as a pure controller.
• –The official RPC docs describe the backend as proof-of-concept and insecure on open networks, so the convenience/perf trade-off comes with real deployment caveats.
• –Community testing around Strix Halo suggests 10GbE or Thunderbolt can be “good enough” for usable cluster inference, but better links mostly reduce overhead instead of changing the basic scaling model.
• –If the model already fits on one machine, single-node inference is usually faster; distributed setup mainly buys you the ability to run larger models that would not otherwise fit.
• –For more tokens per second, the bigger levers are usually model choice, quantization, batching, and parallel request settings, not trying to force every node to 100% utilization.