
BX
Boyang Xu, Shengzhe Chen, Hao Yan
· 1 min read
ResearcharXiv cs.LG
Retimed Bellman Flows: Escaping the Impossible Triangle of Velocity Bootstrapping
arXiv:2609.32828v1 Announce Type: new
Abstract: Flow critics learn return distributions by transporting Gaussian noise to Bellman endpoints via continuous velocity fields. While velocity bootstrapping stabilizes training by querying a successor teacher, existing methods face a structural dilemma: on straight paths, no residual-free same-time affine mapping can preserve Gaussian initial noise while maintaining an unbiased target. To overcome this limitation, we introduce Retimed Bellman Flows (ReBF). ReBF queries the teacher critic at a dynamically shifted earlier flow time, aligning intermediate student and teacher trajectories. By combining this retimed clock with fresh, decoupled noise generation, ReBF constructs a provably conditionally unbiased velocity target that preserves the Bellman fixed point and contracts under Wasserstein distances. Empirically, ReBF reduces $W_1$ distance to ground-truth return distributions by up to $7.7\times$ on synthetic MRPs and outperforms existing flow critics across 38 challenging OGBench and D4RL offline reinforcement learning tasks.
Original source
This story was published by arXiv cs.LG and written by Boyang Xu, Shengzhe Chen, Hao Yan. SyncAI.news shows a preview; the complete article is on the publisher's site.
Read the full story on arxiv.org


