
ZW
Zian Wang, Pu Lu, Caishuang Huang, Wenxi Wu, Meng Zhou, Xiku Du, Yang Qiao
· 1 min read
ResearcharXiv cs.CL
Criterion-Activated Preference Synthesis: Attribution-Preserving Preference Data from Explicit Standards
arXiv:2611.18663v1 Announce Type: new
Abstract: Direct Preference Optimization (DPO) is notoriously sensitive to attribution ambiguity: when chosen and rejected responses differ along multiple entangled dimensions, the model cannot identify which behavior the preference label teaches. We propose Criterion-Activated Preference Synthesis (CAPS), a criterion-first framework that turns explicit quality standards into attribution-verified preference pairs. CAPS synthesizes contexts and responses from a target criterion, then applies dual-signal scoring: a rubric-bound judge measures criterion compliance, while a blind general judge measures holistic quality. Two filters close the attribution loop: the Empirical Activation Test (EAT) keeps criterion-context pairs where satisfying the criterion improves blind quality, and the Attribution Ratio (AR) keeps pairs whose quality gap is dominated by the target criterion. Instantiated with general rubrics, medical rubrics, and Chinese legal statutes, CAPS shows strong data efficiency and domain transfer. With 20k general synthetic pairs, it outperforms 35k human-annotated HelpSteer3 pairs on IFEval, IFBench, AlpacaEval, and Creative Writing v3. In domain settings, CAPS improves HealthBench with 2k medical pairs and improves Qwen3-8B/14B by +4.9 to +6.6 points on legal benchmarks using only public statutes, without human preference labels. We additionally construct the CAPS-Legal preference dataset.
Original source
This story was published by arXiv cs.CL and written by Zian Wang, Pu Lu, Caishuang Huang, Wenxi Wu, Meng Zhou, Xiku Du, Yang Qiao. SyncAI.news shows a preview; the complete article is on the publisher's site.
Read the full story on arxiv.org


