SyncAI.news, a Varaisys broadcasting
Look Before You Select: Rethinking Vocabulary Sparsification in On-Policy Distillation
YM

Yongliang Miao, Shuang Liu, Yanguang Liu, Yandong Bai, Mengnan Du

· 1 min read

ResearcharXiv cs.CL

Look Before You Select: Rethinking Vocabulary Sparsification in On-Policy Distillation

arXiv:2609.34386v1 Announce Type: new Abstract: On-policy distillation (OPD) uses teacher correction on student-generated responses. Full-vocabulary correction can provide important corrections even for tokens that the student assigns low probability, but backpropagating through all token logits becomes memory-intensive for long sequences. Existing memory-saving approaches estimate corrections from sampled tokens or restrict supervision to the student's TopK tokens, introducing sampling noise or changing the full-vocabulary correction. We introduce \textbf{SparseOPD}, which uses full-vocabulary teacher correction to determine which corrections matter before selecting the token logits to differentiate. SparseOPD first constructs the full-vocabulary correction without retaining its backward graph, then selects tokens by correction magnitude rather than student probability. Signed residual compensation preserves the total promoting and suppressing correction mass, while correction-aware budget allocation distributes the sparse support across positions. Finally, the update backpropagates only through the selected token logits. Across six task--scale settings spanning mathematics, chemistry QA, and multimodal reasoning, SparseOPD outperforms Sampled Token and TopK in task-average accuracy and matches or exceeds Full Vocabulary. Gradient cosine similarity reaches 99\% on 4B mathematics, while 8K full-parameter profiling shows 70.5\% lower backward memory.

Original source

This story was published by arXiv cs.CL and written by Yongliang Miao, Shuang Liu, Yanguang Liu, Yandong Bai, Mengnan Du. SyncAI.news shows a preview; the complete article is on the publisher's site.

Read the full story on arxiv.org

Similar News