Open WebUI RAG Eats VRAM, Model Hits RAM

This reads less like “the model is too big” and more like Open WebUI is inserting its own local inference path into the request chain, which can steal GPU resources from the actual model server.

• –Open WebUI’s docs explicitly say it loads local ML models for RAG and STT, and recommend offloading embeddings to an external service to avoid RAM pressure.
• –The project also documents that OpenAI-compatible backends like LM Studio are supported, but that only covers the chat API contract, not how Open WebUI handles its own retrieval pipeline.
• –There’s prior community reporting that RAG can reserve 2 to 4 GB of VRAM inside Open WebUI, which matches the “2 GB of something else” symptom described here.
• –If disabling RAG truly changes nothing, the next likely culprit is the Open WebUI Docker image or embedding configuration still using a local CUDA-backed path.
• –Net: this looks like a real integration quirk, not user incompetence, and it’s exactly the kind of VRAM contention that makes self-hosted AI stacks feel brittle.