Librarian drops 125M SLM series and SFT framework
Developer Sujal Maheshwari has released the Librarian series, a collection of 125M parameter language models trained from scratch using a custom 16k BPE tokenizer. This release includes base and instruct variants alongside Librarian-SFT, a modular, config-driven framework for supervised fine-tuning on consumer-grade hardware.
Librarian proves that training from scratch isn't just for big labs, it's the ultimate "build vs. buy" flex for SLM researchers. Modern architecture like RoPE and SwiGLU makes it a superior baseline for experimentation compared to aging GPT-2 checkpoints, while the custom 16k BPE tokenizer provides a clean slate for testing domain-specific vocabularies. These sub-1B models are becoming the essential building blocks for low-latency "micro-agents" in complex on-device workflows, and the inclusion of the full training and SFT pipeline sets a high standard for transparency and community reproducibility.
DISCOVERED
1d ago
2026-04-14
PUBLISHED
1d ago
2026-04-13
RELEVANCE
AUTHOR
Kill_Streak308