SyncAI.news, a Varaisys broadcasting
On-policy distillation as a Backdoor Propagation Channel
ZF

Zane Fu, Yanzhao Zheng, Ziting Qin, Qianshan Wei, Yi Yu, Gongli Xi, Jihuai Zhu, Baohua Dong, Hangcheng Zhu, Ruohui Huang, Gang Yu

· 1 min read

ResearcharXiv cs.LG

On-policy distillation as a Backdoor Propagation Channel

arXiv:2611.19207v1 Announce Type: new Abstract: On-policy distillation (OPD) is emerging as an important paradigm for the post training of large language models, improving student models by leveraging supervision generated online by teacher models. However, whether this process also transfers latent security risks from teacher models to student models remains insufficiently studied. In this work, we reveal a new security risk in which an attacker first implants a conditional backdoor into the teacher model through supervised fine tuning (SFT), after which OPD can transfer the backdoor to the student model with only an extremely small proportion of trigger prompt exposure. More concerningly, we find that such backdoor transfer is substantially stronger under OPD than under conventional distillation methods, suggesting that OPD may provide an efficient channel for propagating backdoors across models. Our mechanistic analysis shows that the backdoor signal is primarily transmitted through anomalous differences in the conditional probabilities produced by the teacher model for triggered and clean inputs, and is gradually internalized into the internal representations of the student model during training. Furthermore, we investigate how to suppress such anomalous conditional probability signals at the source of distillation supervision and, based on this analysis, propose an OPD training method tailored to this conditional backdoor propagation setting to mitigate the risk of backdoor transfer induced by untrusted teacher supervision.

Original source

This story was published by arXiv cs.LG and written by Zane Fu, Yanzhao Zheng, Ziting Qin, Qianshan Wei, Yi Yu, Gongli Xi, Jihuai Zhu, Baohua Dong, Hangcheng Zhu, Ruohui Huang, Gang Yu. SyncAI.news shows a preview; the complete article is on the publisher's site.

Read the full story on arxiv.org

Similar News