Route to Remember Makes Weight Memory Practical

The important result is not that fine-tuning automatically beats RAG; it is that the training objective determines whether weight memory works at all.

• –On-policy self-distillation reached 59.9% accuracy, while plain SFT on identical data managed only 36.9%.
• –Weight memory sharply reduces read-time context and token costs, but shifts expense into memory-writing and training.
• –The hybrid router improves five of six question categories, showing retrieval remains valuable for conflicts, updates, and uncertain facts.
• –Weight memory gradually forgets older information and can confidently invent answers when a fact was never recorded.
• –For production agents, the strongest architecture looks hybrid: internalize stable facts, retrieve volatile evidence, and verify before responding.