Dual-Anchored Policy Distillation solves the privilege illusion in LLM post-training by aligning student and teacher information paths.
Dual-Anchored Policy Distillation (DAPD) is a post-training framework designed to tackle the "privilege illusion" in on-policy self-distillation (OPSD). When teacher models rely on reference completions or ground-truth context unavailable to students during inference, performance drops significantly upon privilege removal. DAPD resolves this asymmetry by combining Dual-Path Anchoring (DPA) and Dual-Source Anchoring (DSA) to enforce bidirectional supervision across matched-information paths. Empirical testing across the Qwen3 model family shows DAPD consistently outperforms traditional OPSD by up to +2.78 points.
The "privilege illusion" is a widespread issue in LLM post-training alignment, and DAPD offers a structural fix to ensure student models perform robustly without ungranted inference privileges.
* Eliminates student reliance on privileged teacher context during on-policy distillation.
* Employs Dual-Path and Dual-Source Anchoring to guarantee information parity across rollout paths.
* Achieves scalable gains across Qwen3 model benchmarks ranging from 1.7B to 32B parameters.
DISCOVERED
1h ago
2026-08-04
PUBLISHED
1h ago
2026-08-04
RELEVANCE
AUTHOR
_akhaliq