SyncAI.news, a Varaisys broadcasting
The Schema Coherence Inventory: Measuring Whether an Assigned Belief Organizes a Language Model's Cognition
YW

Yuqi Wu, Deyang Li, Katharine Hibbard, Ron Oswald, David Sinclair, Adegboyega Sapara, Kalutota Kuruwe Arachchige Sajana Indula Samarasinghe, Abhinav Pillai, Reham Shalaby, Riffat Alam, Xin-Min Li, Yanbo Zhang, Jie Chen

· 1 min read

ResearcharXiv cs.CL

The Schema Coherence Inventory: Measuring Whether an Assigned Belief Organizes a Language Model's Cognition

arXiv:2610.31683v1 Announce Type: new Abstract: Persona-conditioned language models are used as simulated patients and affective agents, but there is no validated way to ask whether a belief assigned to such a model organizes its behaviour or whether the model has merely adopted a mood. We present the Schema Coherence Inventory (SCI): 96 forced-choice items at eight levels of Beck's cognitive model, each offering six replies keyed to three negative self-schemas and their positive counterparts, scored by length-normalized log-likelihood so that nothing is generated or parsed. Affect alone can move a model onto a default belief, so the discriminant criterion is target specificity: a condition built for one belief must select that belief more often than conditions built for the other beliefs of its pole. Six validation studies over 107 conditions on five backbones (1.7B-14B parameters) support this reading. Nine clinically trained raters recover the keyed belief at 79.5% (chance 16.7%). Belief adapters are target-specific on every backbone (+17.7 points; 30/30 belief cells positive), and a condition's SCI profile identifies its own free generations in 24/30 cases against 5 expected by chance. A 48-item short form correlates r = 0.73 with the other 48 items. Applied to six published patient simulators, none carries its formulation's core belief better than a one-sentence statement of it. The validation route, which separates a construct from the affect that accompanies it and checks model judges against clinicians, offers a template for affective measurement in language models; we outline its extension across task types, raters, sites and languages. We release the items, scoring code and short form.

Original source

This story was published by arXiv cs.CL and written by Yuqi Wu, Deyang Li, Katharine Hibbard, Ron Oswald, David Sinclair, Adegboyega Sapara, Kalutota Kuruwe Arachchige Sajana Indula Samarasinghe, Abhinav Pillai, Reham Shalaby, Riffat Alam, Xin-Min Li, Yanbo Zhang, Jie Chen. SyncAI.news shows a preview; the complete article is on the publisher's site.

Read the full story on arxiv.org

Similar News