Memory Tax Finds RAG Beats Agent Memory
The paper evaluates 11 agent-memory systems across LongMemEval, PersonaMem-v2, and MINT-Eval, finding many underperform naive RAG over raw history while costing substantially more. It introduces “memory tax” to measure evidence lost or obscured during memory construction. [Paper](https://arxiv.org/abs/2610.34073) [Project page](https://hexuan21.github.io/MemoryTax/)
The hot take is simple: sophisticated memory layers need to justify their complexity against a strong raw-history retrieval baseline.
- –Memory processing can discard, fragment, dilute, or rephrase evidence that RAG preserves.
- –The study reports 90 of 99 memory results falling below naive RAG.
- –Memory systems can add significant indexing and LLM-call costs without delivering consistent accuracy gains.
- –The right metric is net value: memory benefit minus memory tax, not standalone benchmark score.
- –Source-history augmentation offers a practical recovery path for systems that preserve useful evidence but retrieve it poorly.
DISCOVERED
1h ago
2026-10-27
PUBLISHED
2h ago
2026-10-27
RELEVANCE
AUTHOR
omarsar0