llama.cpp OpenVINO backend tops SYCL on Intel

The big story is that OpenVINO makes llama.cpp feel competitive on Intel hardware for the first time, but the victory is partial because Intel’s tuned stack still wins where it matters most for real workloads.

• –OpenVINO posts roughly 4.5x higher prompt-processing throughput than SYCL in this benchmark, which is the clearest signal that the new backend is a real step forward.
• –LLM-Scaler still leads overall on the same class of model, likely because Intel has spent more effort optimizing around GPTQ/Int4 and its own serving path.
• –Token generation tells a different story: SYCL was faster on tg512, so backend choice still depends on the mix of prefill versus decode you care about.
• –The model-compatibility caveat matters as much as the speedup; Intel’s validated model lists remain a practical bottleneck for anyone trying to reproduce results.
• –This is less “winner take all” than “Intel finally has multiple viable inference stacks,” which is progress, but also a sign the ecosystem is still fragmented.