SyncAI.news, a Varaisys broadcasting
Lost in Read-out: Locating the Image Quality Gap in Vision-Language Models with a Closed-Form Edit
ZL

Zhuangzi Li, Shilv Cai, Jian Jin, Zhiyi Pan, Haihong Xiao, Weisi Lin

· 1 min read

ResearcharXiv cs.CV

Lost in Read-out: Locating the Image Quality Gap in Vision-Language Models with a Closed-Form Edit

arXiv:2611.22564v1 Announce Type: new Abstract: Vision-language models (VLMs) are widely used as zero-shot image quality assessors, yet their good-vs-poor answers agree only moderately with human opinion. We ask whether these models fail to see quality or fail to say it. Layer-wise probes on five VLMs point to the second: quality is linearly decodable from the vision encoder and still from the last language layer, far better than the answer reveals, and about a hundred labels recover most of it. The loss lies mainly in the final read-out, which partly reads something else: relative to human opinion, it over-rewards colorfulness and under-rewards sharpness on all five models. We test this by intervention: if the read-out is at fault, a closed-form edit of a single last-layer neuron, fitted to many prompts at once, should close the gap and no more. Both hold. Under prompts never used for fitting, the edit recovers 86--100\% of what a ridge head on the last-layer state, fitted under the same prompts, finds beyond the answer on KonIQ. Where that gap is small, it gains little, too little for our transfer criterion on the two largest models; where the head itself is below the answer, it lowers the score; and it works only when written from the middle of the language model on. Unlike unconstrained fine-tuning, the edit leaves perception question answering intact. It does not carry over to yes-or-no or rating questions, and it reverses the good--poor contrast after sentiment-laden sentences. Code will be made available.

Original source

This story was published by arXiv cs.CV and written by Zhuangzi Li, Shilv Cai, Jian Jin, Zhiyi Pan, Haihong Xiao, Weisi Lin. SyncAI.news shows a preview; the complete article is on the publisher's site.

Read the full story on arxiv.org

Similar News