Ollama powers 12K-PDF laptop RAG

Local RAG is moving out of toy-demo territory, and this post is a good snapshot of what “good enough on a laptop” now looks like for developers.

• –The setup uses Ollama with a quantized small model, which is exactly the kind of hardware-conscious stack real developers can afford and replicate
• –The author says the hardest part was parsing, not inference, and that they built their own pipeline to handle PDFs plus other document formats over two years
• –A 12k-document corpus on consumer hardware makes the case for private, offline knowledge systems where cloud upload is a non-starter
• –The comments also surface the next bottleneck: retrieval quality depends as much on embeddings and search strategy as on the local LLM itself