SyncAI.news, a Varaisys broadcasting
Towards VLA-Dreamer: Refining VLA Behavior Using World Models
PM

Parsa Mastouri Kashani, Jan-Gerrit Habekost, Stefan Wermter

· 1 min read

ResearcharXiv cs.AI

Towards VLA-Dreamer: Refining VLA Behavior Using World Models

arXiv:2609.31313v1 Announce Type: cross Abstract: Vision-Language-Action models (VLAs), while showing strong potential for robot control, require massive amounts of high-quality imitation learning data. Moreover, the absence of an explicit world model casts further doubt on their control capabilities. In this concept paper, we propose a novel architecture that addresses sample efficiency in VLAs by training a predictive world model on the embedding space of the VLA's vision encoder. We hypothesize that these embeddings are action-relevant and usable for future prediction. To this end, we propose using the suggested architecture to investigate how well these embeddings predict the future based on actions, as the inability to do so would mark a key limitation of VLA architectures: the lack of a non-lossy implicit world model to simulate real-world dynamics. The proposed architecture differs from the standard world model dynamics as the loss comes from the embedding space rather than the pixel space, similar to joint embedding predictive architectures. Furthermore, the trained world model can be utilized for short-term planning tasks by sampling VLA actions given goal images. We intend to examine the richness of vision embeddings in VLAs and reduce their high data requirements through a world model that can also generate plans during inference.

Original source

This story was published by arXiv cs.AI and written by Parsa Mastouri Kashani, Jan-Gerrit Habekost, Stefan Wermter. SyncAI.news shows a preview; the complete article is on the publisher's site.

Read the full story on arxiv.org

Similar News