Local knowledge system hits 32K docs

This is the kind of practical edge-AI progress that matters more than flashy model launches: better document scale, lower retrieval cost, and consumer hardware that starts to look enterprise-useful. It is still a demo rather than a polished product, but the tradeoffs are getting much more believable for private on-device knowledge systems.

• –The jump from roughly 12K to 32K documents on a $1,299 laptop is a meaningful signal for local-first RAG deployments.
• –Preserving folder structure during indexing matters because it maps better to real enterprise knowledge bases and access-control boundaries.
• –Cutting retrieval payload to about 1,200 tokens makes small local models more viable and keeps latency and cost pressure down.
• –The author says larger models still format answers better, which shows retrieval scale is improving faster than final answer quality on tiny models.