Local LLM Devs Debate Doc-to-LoRA, RAG

Doc-to-LoRA is forcing the local LLM community to rethink long-term memory architectures, moving the debate from "how do we search" to "how do we patch."

• –RAG remains the safe bet for exact quotes and deterministic retrieval, but it clogs up the context window and scales poorly for massive personal archives.
• –Doc-to-LoRA's ability to inject a 128k-token document into a model as a hot-swappable adapter via a single forward pass could eliminate the need for vector databases entirely for some use cases.
• –The concept of generating on-demand "skill LoRAs" from documentation means local agents could dynamically download and apply knowledge patches instead of performing slow web searches.
• –While still experimental, replacing RAG's semantic search with Doc-to-LoRA's hypernetwork-driven context distillation represents the biggest architectural shift in local AI since the introduction of KV-caching.