OPEN_SOURCE ↗
REDDIT · REDDIT// 35d agoOPENSOURCE RELEASE
rho-eval adds snap-on instruct adapters
rho-eval is an open-source LLM auditing and repair toolkit that now packages “snap-on” logit-space adapters and rho-unlock diagnostics for recovering instruction-following behavior on frozen base models. The pitch is unusually practical for local-model tinkerers: the adapters are tiny, claim zero MMLU knowledge loss on tested models, and can be trained in hours on consumer Apple Silicon hardware.
// ANALYSIS
This is a clever reframing of post-training: instead of repeatedly fine-tuning the whole model, rho-eval treats “knowing” and “speaking like an assistant” as separable layers you can swap at inference time.
- –The most interesting claim is zero knowledge damage: the Qwen 7B base reportedly keeps its 57.6% MMLU score while a 29M logit-space adapter boosts instruction-following behavior
- –The project is more than a paper dump; PyPI, a CLI, MLX support, and a GitHub repo make it usable for practitioners who want to experiment without a cluster
- –If the approach holds up beyond the author’s benchmarks, it could become a cheaper alternative to full SFT for domain-specific local assistants and safety tuning
- –The big caveat is validation breadth: most evidence comes from one independent researcher’s repo, Zenodo preprints, and limited model families rather than broad third-party replication
// TAGS
rho-evalllmfine-tuningopen-sourcecliresearch
DISCOVERED
35d ago
2026-03-08
PUBLISHED
35d ago
2026-03-08
RELEVANCE
8/ 10
AUTHOR
NoSir261