AnythingLLM, vLLM stack powers Synology NAS

This reads like the kind of deployment note people end up writing after days of trial-and-error: not glamorous, but genuinely useful if you want local RAG to run reliably on constrained hardware.

• –Separating inference from the UI is the right call here; it keeps AnythingLLM lightweight and lets vLLM do the GPU-heavy work.
• –The version pin to `vllm/vllm-openai:v0.8.5` matters more than the model choice, because CUDA and driver compatibility are the real constraint on Synology-class setups.
• –Using vLLM's OpenAI-compatible API makes AnythingLLM integration straightforward, so the stack stays portable across local and cloud backends.
• –The embedding side is still externalized to Ollama, which means this is really a composed local AI stack rather than a single all-in-one deployment.
• –The post is hardware-specific, but the pattern generalizes well: shared network, explicit volume permissions, and conservative image pinning are the pieces that keep self-hosted LLM stacks from breaking.