Princeton paper turns knowledge graphs into reward models

This is a persuasive case for structured supervision: if the domain already has a graph, you can reward the reasoning process itself instead of guessing at preferences from final answers.

• –Path-level rewards are a much cleaner training signal than raw outcome-only RL for domains like medicine, where intermediate steps can be checked against facts.
• –The big result is zero-shot transfer from 1-3 hop training to 4-5 hop questions, which is the kind of compositional leap most post-training recipes struggle to deliver.
• –The paper’s robustness claims on option shuffling matter because they suggest the model is learning structure, not just exploiting answer-position shortcuts.
• –The caveat is obvious: this approach is only as good as the knowledge graph, so incomplete or noisy graphs can become incomplete or noisy rewards.
• –Open-sourcing the pipeline makes this interesting beyond the paper itself, because teams can test whether KG-grounded rewards hold up outside the biomedical setting.