SyncAI.news, a Varaisys broadcasting
Is Sycophancy a Form of In-Context Updating?
IS

Itai Shapira, Myra Cheng, Hadas Orgad

· 1 min read

ResearcharXiv cs.AI

Is Sycophancy a Form of In-Context Updating?

arXiv:2610.28325v1 Announce Type: new Abstract: Factual sycophancy, a language model's deference to a user's false beliefs, is usually understood as a bid for the user's approval at the expense of truthfulness. We argue that this framing over-emphasizes the user and propose viewing sycophancy as a failure within the broader and generally useful process of in-context updating, by which models revise their answers based on information supplied during an interaction. We support this view with behavioral and mechanistic evidence. First, models' responses to user claims sit within a broader pattern of source-sensitive updating. They revise their answers in response to information attributed to external sources as well, more so when it appears better supported, and their deference to the user persists even when no user approval is at stake. Second, the internal components that support deference to the user are also causally important for in-context updating more broadly. Our results suggest that factual sycophancy arises when in-context updating gives a user's assertion excessive weight relative to its evidence. Because sycophancy is embedded in this process, reducing it risks reducing useful updating as well, a trade-off that calls for evaluations measuring both behaviors and mitigations that decouple them.

Original source

This story was published by arXiv cs.AI and written by Itai Shapira, Myra Cheng, Hadas Orgad. SyncAI.news shows a preview; the complete article is on the publisher's site.

Read the full story on arxiv.org

Similar News