BACK_TO_FEEDAICRIER_2
llama.cpp SYCL fix stops dual Arc RAM bloat
OPEN_SOURCE ↗
REDDIT · REDDIT// 4d agoNEWS

llama.cpp SYCL fix stops dual Arc RAM bloat

Dual Intel Arc GPUs can drive llama.cpp's SYCL backend into mirroring device allocations in system RAM, which makes hosts OOM long before VRAM is full. The reported fix swaps `sycl::malloc_device()` for Level Zero `zeMemAllocDevice()` with fallback behavior, keeping host memory flat without hurting inference speed.

// ANALYSIS

This looks like a driver-path trap, not a tuning problem: the model fit was never the issue, the allocation API was. If the fix upstreams cleanly, it removes a major blocker for practical multi-GPU Intel local inference.

  • `sycl::malloc_device()` appears to hit xe's DMA-buf/TTM path, so GPU allocations get mirrored into host RAM at allocation time
  • `zeMemAllocDevice()` avoids that mirror behavior, and the post claims SYCL kernels can still read the pointers without issues
  • The failure mode is nasty because kernel-side staging bypasses cgroup limits and looks like an application-level memory leak
  • The evidence points to a stability win more than a performance win: throughput stays flat while RAM usage drops from crash territory to roughly 10%
  • If reproducible broadly, this changes the hardware advice for dual Intel Arc setups from "buy more system RAM" to "use the right allocator"
// TAGS
llama-cppinferencegpuopen-sourcellm

DISCOVERED

4d ago

2026-04-08

PUBLISHED

4d ago

2026-04-08

RELEVANCE

8/ 10

AUTHOR

Katostrofik