Self-Patching bridges LLM knowing-using gap
Researchers from HKUST have introduced Self-Patching, a novel mechanistic interpretability technique designed to locate and resolve routing failures in fine-tuned Large Language Models. By relocating internal representations of memorized facts, the technique bridges the "Knowing-Using Gap" and recovers 58% to 75% of the reasoning generalization headroom.
This paper shifts the SFT/LoRA failure narrative from model capacity to information routing, proving LLMs "know" the facts but simply lose the path to compute them.
* Mechanistic interpretability is moving from passive observation to active intervention, showing we can patch representation flow to bypass fine-tuning limitations.
* If relocating activation states can restore 75% of generalization headroom, inference-time activation patching could become a viable alternative to costly reinforcement learning or multi-stage alignment.
* The "Knowing-Using Gap" highlights that current fine-tuning methods (SFT, LoRA) are highly inefficient at circuit alignment, raising questions about whether they are fundamentally flawed for complex reasoning tasks.
DISCOVERED
1h ago
2026-07-12
PUBLISHED
1h ago
2026-07-12
RELEVANCE
AUTHOR
Discover AI