SyncAI.news, a Varaisys broadcasting
VidToC: Rethinking Video Token Compression for Multimodal Large Language Models
ZX

Zihan Xiao (AIM3 Lab, Renmin University of China), Boshen Xu (AIM3 Lab, Renmin University of China), Pei Fu (MiLM Plus, Xiaomi Inc), Jianzhong Ju (MiLM Plus, Xiaomi Inc), Zhenbo Luo (MiLM Plus, Xiaomi Inc), Jian Luan (MiLM Plus, Xiaomi Inc), Qin Jin (AIM3 Lab, Renmin University of China)

· 1 min read

ResearcharXiv cs.CV

VidToC: Rethinking Video Token Compression for Multimodal Large Language Models

arXiv:2611.25574v1 Announce Type: new Abstract: Extending multimodal large language models (MLLMs) from images to videos faces substantial memory and latency challenges due to the massive number of visual tokens. These challenges have driven the development of numerous visual token compression methods and more efficient LLM architectures. However, existing compression methods are often evaluated with different training setups, tasks, and models, making fair comparison difficult. Meanwhile, the growing use of hybrid Mamba-Transformer architectures raises questions about whether compression strategies transfer across backbones. We present VidToC, a controlled evaluation framework for representative compression algorithms, LLM backbones, and time encoding strategies, covering training-free and fine-tuned settings on three complementary video understanding tasks. Our results reveal task-dependent compression behavior and characterize the effects of compression placement, time encoding, and backbone choice. These findings provide empirical guidance for designing and selecting video token compression strategies across tasks and training settings.

Original source

This story was published by arXiv cs.CV and written by Zihan Xiao (AIM3 Lab, Renmin University of China), Boshen Xu (AIM3 Lab, Renmin University of China), Pei Fu (MiLM Plus, Xiaomi Inc), Jianzhong Ju (MiLM Plus, Xiaomi Inc), Zhenbo Luo (MiLM Plus, Xiaomi Inc), Jian Luan (MiLM Plus, Xiaomi Inc), Qin Jin (AIM3 Lab, Renmin University of China). SyncAI.news shows a preview; the complete article is on the publisher's site.

Read the full story on arxiv.org

Similar News