
MQ
Meng'en Qin, Junye Chen, Jucheng Liu, Yinchen Liu, Youlu Xing, Song Wang, Ruize Han
· 1 min read
ResearcharXiv cs.CL
MLLMs Hallucinate when Information Distribution Drifts in Synergy Heads
arXiv:2609.09206v2 Announce Type: replace-cross
Abstract: Multimodal Large Language Models (MLLMs) often struggle with hallucinations, thus hindering their reliable practical applications. Existing attention-based mitigation methods mainly rely on indirect signals (e.g., attention weights) that fail to accurately reflect the actual information shift underlying hallucination generation. In this paper, we propose HEAL, Head-lEvel information disentAnglement and caLibration for identifying and mitigating hallucinations. HEAL first employs causal noise intervention on multi-head outputs to filter out causally redundant heads. Subsequently, it disentangles information distribution within the remaining heads via the counterfactual Difference-in-Differences, categorizing heads into four types. Through analysis, we observe: hallucinations happen when information distribution drifts away from a healthy equilibrium in synergy heads, not strongly correlated with the quantity or strength of modality-specific heads. Motivated by this insight, HEAL injects dynamic information calibration factors into the value vectors of synergy heads and actively regulates visual-language dependencies, steering the output distribution towards factual evidence. Extensive experiments demonstrate that HEAL effectively reduces hallucinations across multiple MLLMs, offering a simple and interpretable pathway to enhance model trustworthiness.
Original source
This story was published by arXiv cs.CL and written by Meng'en Qin, Junye Chen, Jucheng Liu, Yinchen Liu, Youlu Xing, Song Wang, Ruize Han. SyncAI.news shows a preview; the complete article is on the publisher's site.
Read the full story on arxiv.org


