OPEN LINK ↗
// 27m agoRESEARCH PAPER
Deletion Tuning Matches Full Fine-Tuning
The Deletion Tuning paper shows that zeroing fewer than 0.05% of an LLM’s weights can match full fine-tuning across reinforcement learning, supervised fine-tuning, and self-distillation. The learned binary mask is dramatically smaller and reduces catastrophic forgetting.
// ANALYSIS
This is a compelling challenge to the assumption that adaptation requires writing new values into model weights, though the method remains research-grade.
- –Learns which gate-projection weights to delete rather than adding LoRA-style parameters
- –Matches full fine-tuning across multiple training regimes and model sizes
- –Produces masks hundreds to thousands of times smaller than full checkpoints
- –Preserves prior-task performance better, with competing methods losing up to 5.4 points under reinforcement learning
- –Public code and pretrained masks make the result unusually easy to reproduce
// TAGS
deletion-tuningllmfine-tuningtrainingopen-sourceresearch
DISCOVERED
27m ago
2026-11-10
PUBLISHED
1h ago
2026-11-10
RELEVANCE
9/ 10
AUTHOR
0x0SojalSec