llama.cpp benchmarks validate cross-vendor eGPU inference

This deep dive into heterogeneous inference confirms that memory bandwidth, not PCIe constraints, dictates local LLM performance.

• –Vulkan abstracts the hardware layer to stably combine AMD and NVIDIA architectures with only a 5-10% performance penalty compared to native CUDA or ROCm.
• –OCuLink bandwidth uses less than 1% of its capacity during active token generation, as only tiny activation tensors are passed between GPUs.
• –Offloading layers to slower system memory causes a non-linear performance drop governed by Amdahl's Law, creating pipeline stalls as the fast eGPU waits for the APU.
• –The results pave the way for building massive, high-capacity local inference rigs by pooling cheap unified APU memory with fast dedicated eGPU VRAM.