SyncAI.news, a Varaisys broadcasting
When Sounds Turn Heads: Recovering Event Timing That Audio LLMs Fail to Say
JW

Jongbhin Woo, Jaewook Jeong, Joon Son Chung

· 1 min read

ResearcharXiv cs.CL

When Sounds Turn Heads: Recovering Event Timing That Audio LLMs Fail to Say

arXiv:2611.25960v1 Announce Type: cross Abstract: Audio LLMs can know when a queried sound occurs but fail to say it. We find a few attention heads that track the queried sound: they shift when the query changes, follow the sound when it moves, and removing or blocking them often hurts the model's timestamps. Yet the answer does not express what they track. Intervals read directly from these heads outperform the model's original answer and recover events that its timestamps miss. To bring this timing into the answer, we introduce head-guided decoding, which needs no training. It keeps the model's decision on whether the sound is present and rescores its candidate timestamps with a prior from these heads. Heads selected on development data transfer unchanged to new recordings and free-text queries, where head-guided decoding improves the timestamps of three audio LLMs, scores above their head readout on every benchmark, and, in most settings, outperforms random-head guidance. On recordings released after the models were trained, head-guided decoding raises mIoU from 0.07-0.36 to 0.48-0.56. These results suggest that poor temporal answers in audio LLMs partly reflect a failure to say rather than to hear, and that the model's own attention can close part of this gap.

Original source

This story was published by arXiv cs.CL and written by Jongbhin Woo, Jaewook Jeong, Joon Son Chung. SyncAI.news shows a preview; the complete article is on the publisher's site.

Read the full story on arxiv.org

Similar News