SyncAI.news, a Varaisys broadcasting
MotionGrounder: Grounded Multi-Object Motion Transfer via Diffusion Transformer
ST

Samuel Teodoro, Yun Chen, Agus Gunawan, Soo Ye Kim, Jihyong Oh, Munchurl Kim

· 1 min read

ResearcharXiv cs.CV

MotionGrounder: Grounded Multi-Object Motion Transfer via Diffusion Transformer

arXiv:2604.00853v2 Announce Type: replace Abstract: Motion transfer enables controllable video generation by transferring temporal dynamics from a reference video to synthesize a new video conditioned on a target caption. However, existing Diffusion Transformer (DiT)-based methods either transfer the motion of a single object or handle multiple objects without binding each object caption to its object, restricting fine-grained control in real-world scenes. In this work, we introduce MotionGrounder, a training-free DiT-based framework that is the first to jointly handle motion transfer and object grounding in multi-object scenes. Our Flow-based Motion Signal (FMS) in MotionGrounder provides a stable motion prior for target video generation, while our Object-Caption Alignment Loss (OCAL) grounds object captions to their corresponding spatial regions. We further propose two new metrics: the Object Grounding Score (OGS), which jointly evaluates (i) spatial alignment between source video objects and their generated counterparts and (ii) semantic consistency between each generated object and its target caption, and the Motion Grounding Score (MGS), which evaluates whether each object moves like its source object at the correct location. Our experiments show that MotionGrounder improves object grounding over recent baselines, with the advantage appearing once several objects are present, across quantitative, qualitative, and human evaluations.

Original source

This story was published by arXiv cs.CV and written by Samuel Teodoro, Yun Chen, Agus Gunawan, Soo Ye Kim, Jihyong Oh, Munchurl Kim. SyncAI.news shows a preview; the complete article is on the publisher's site.

Read the full story on arxiv.org

Similar News