Frank Bot shares hard-won RAG lessons

Regulated RAG is mostly a systems problem wearing an ML costume. The model matters, but retrieval routing, prompt boundaries, and tenant isolation are what make the system reliable enough to ship.

• –Four query rewrites via Claude Haiku improve recall more than fiddling with chunk size alone, especially when users and policy docs use different jargon.
• –Named-document boosting is a smart compliance hack because the right answer often lives in one obvious source of truth, not the nearest semantic neighbor.
• –Immutable Layer 1 prompts are the real guardrail in multi-tenant bots; without them, client instructions become an attack surface.
• –Local `all-MiniLM-L6-v2` embeddings plus ChromaDB are a sane tradeoff when the LLM and retrieval logic do the heavy lifting.
• –One VM per client is operationally boring in the best way: simpler isolation, fewer cross-contamination risks, and less shared-state pain.