Finetuning with Sampling Makes SFT Rival RL

The clever move is distribution engineering: preserve expert information while making training data more native to the learner, challenging the assumption that RL inherently has the generalization advantage.

• –MCMC progressively refines expert traces using base-model likelihoods while preserving task-relevant constraints.
• –Experiments compare against GRPO, UFT, and OPSD across scientific skill acquisition, reasoning, and open-ended expertise.
• –The method reportedly learns capabilities beyond simple distribution sharpening, including tasks the base model cannot reliably solve.
• –Extra sampling and likelihood-scoring steps add compute and implementation complexity, so the practical advantage over RL remains to be tested at scale.
• –If validated broadly, projection sampling could become a reusable post-training primitive for SFT, RL, and distillation.