llama.cpp RPC benchmarks favor Linux, 2.5GbE

This is less a launch story than a reality check: llama.cpp RPC works, but the gains are sensitive to OS, driver stack, and network quality.

• –Native Linux outperformed Windows and WSL in these runs, which lines up with the usual overhead and networking quirks around WSL
• –The jump from 1GbE to 2.5GbE helped, but the reported traffic levels suggest the workload is not constantly saturating the link
• –The post reinforces that RPC is practical for smaller contexts and mixed-GPU home labs, not a free way to scale arbitrarily
• –The author’s note about flash attention slowing things down is a reminder that “more features” can hurt on consumer hardware if the config is not tuned
• –If the goal is squeezing larger contexts across multiple machines, this kind of setup still looks promising, but it is clearly closer to enthusiast infrastructure than plug-and-play inference