Llama.cpp VLM batching hits CUDA OOM

While llama.cpp remains the gold standard for local inference, this report proves that production-grade reliability for batch multimodal tasks still requires refinement.

• –**Long-tail stability:** Processing 15,000 images is a rigorous stress test that standard benchmarks often fail to capture.
• –**CUDA resource leaks:** The consistent failure threshold suggests that GPU memory buffers or inference contexts are not being properly recycled within the server loop.
• –**Production bottleneck:** As developers transition from chat experiments to autonomous batch processing, server-side memory management becomes a critical reliability factor.
• –**Interim workarounds:** Until a core fix is merged, developers running large-scale vision tasks may need to implement periodic server restarts to clear the CUDA context.