OPEN_SOURCE ↗
REDDIT · REDDIT// 4d agoRESEARCH PAPER
LLM reasoning, forgetting share root cause
Researcher Akihito Sunagawa proposes a "Minimal Model of Structural Persistence" framework identifying the accumulation of unresolved contradictions as the primary driver behind both long-context reasoning degradation and catastrophic forgetting. This shift in perspective moves beyond token limits, suggesting that "drift" is a failure to maintain structural integrity as premises are updated.
// ANALYSIS
This research reframes LLM "forgetting" as an architectural inability to reorganize dependent knowledge, essentially turning learning into an overwrite process.
- –An "External Metabolism Pipeline" organizing contradictions by time boosted logical consistency from 21.1% to 73.3% in long-turn dialogues.
- –"Structural Forgetting" describes the collapse of entire knowledge chains when a single underlying premise is modified during fine-tuning.
- –LoRA-based updates behave like overwriting rather than cumulative learning across model sizes up to 72B.
- –The framework mathematically models "Structural Persistence Potential" as an exponential decay driven by the log-ratio of state space reduction.
// TAGS
llmreasoningfine-tuningresearchcatastrophic-forgettinglora
DISCOVERED
4d ago
2026-04-15
PUBLISHED
4d ago
2026-04-15
RELEVANCE
8/ 10
AUTHOR
IndividualBluebird80