SyncAI.news, a Varaisys broadcasting
VidEngram: Agentic Long-Term Video Understanding via Structured Hierarchical Memory
ZC

Zinuo Cheng, Yueqian Lin, Zhongxi Wang, Hai "Helen" Li, Yiran Chen

· 1 min read

ResearcharXiv cs.CV

VidEngram: Agentic Long-Term Video Understanding via Structured Hierarchical Memory

arXiv:2611.18578v1 Announce Type: new Abstract: Long-video question answering is wasteful and lossy when a multimodal model must re-encode the same video for every query: context limits force sparse sampling or truncation, and no reusable representation remains for later questions. We present VidEngram, an agentic video QA system that builds a persistent hierarchical memory once at ingest and answers later questions from that memory. VidEngram captions video and audio jointly with Qwen2.5-Omni, consolidates segment memories into episode and entity layers, and uses a text-only agentic planner with specialized retrieval tools to produce grounded answers. The web interface makes this process inspectable: users can watch memory formation, inspect retrieval traces, verify cited evidence in the video player, and explore entity relationships. On the full OmniVideoBench, VidEngram with a GPT-5-mini planner outperforms a strong single-pass Qwen2.5-Omni baseline (37.0% vs. 32.3%), with gains concentrated on longer videos where single-pass coverage becomes sparse. Because perception is paid once, subsequent queries use low-cost text-only planning.

Original source

This story was published by arXiv cs.CV and written by Zinuo Cheng, Yueqian Lin, Zhongxi Wang, Hai "Helen" Li, Yiran Chen. SyncAI.news shows a preview; the complete article is on the publisher's site.

Read the full story on arxiv.org

Similar News