Open WebUI, Ollama for Local LLMs

The architecture is directionally right, but the backend choice matters far more than the UI. Open WebUI is a reasonable front end for local chat and RAG; Ollama is fine for prototyping, while vLLM is the better production path once concurrency and throughput start to matter.

• –Open WebUI’s file-context RAG fits the PDF/document use case, but keep ingestion and access controls simple if the goal is strict on-prem privacy
• –Ollama is a low-friction way to test models on a workstation, not the strongest choice for shared multi-user serving
• –vLLM is built around an OpenAI-compatible HTTP server and is the more sensible inference layer for a small internal team
• –The hard constraint is hardware, not the interface: 27B-class models are already demanding, and 122B-class models will push latency and memory hard in practice
• –For a small company, the safest rollout is local-only networking, no tools/agents, and a narrow document workflow before adding more automation