PlurPO Trains LLMs Against Social Sycophancy

PlurPO is a compelling reframing of alignment: social advice needs perspective-taking, not merely user satisfaction. The results are promising, but the method remains sensitive to model capabilities and prompt tuning.

• –A frozen supervisor identifies stakeholders, samples candidate responses, and uses simulated vetoes to create preference pairs.
• –It reduces the gap between model and human endorsement rates on general advice from 17.8% to 8.0%.
• –Preference data generated for an 8B model transfers effectively to a 32B model.
• –The procedure is not plug-and-play: the authors report that an untuned setup failed on Llama 3.1 8B.
• –As an unrefereed preprint, its gains still need independent replication beyond the authors’ evaluations.