SyncAI.news, a Varaisys broadcasting
An analysis of Mirror-Descent Soft Actor-Critic
DZ

Denis Zorba, Michal Valko

· 1 min read

ResearcharXiv cs.LG

An analysis of Mirror-Descent Soft Actor-Critic

arXiv:2609.35466v1 Announce Type: new Abstract: Soft Actor-Critic (SAC) is widely used for entropy-regularised reinforcement learning with continuous action spaces, and practical implementations perform only a few actor steps towards an evolving target. In this work, we prove convergence guarantees when the target policy arises from policy mirror descent and compare it with the classical Gibbs target. We derive sufficient conditions for the strong convexity and smoothness of the actor objective, characterised by the curvature of the $Q$-function estimate through the Legendre differential operator, and establish an $\mathcal{O}\!\left(N^{-\frac{1}{5}}\right)$ best-iterate finite-time convergence rate up to actor and critic approximation errors. Moreover, the mirror-descent step size $\lambda$ directly controls the target drift and hence actor tracking error, whereas the analogous Gibbs bound contains a non-vanishing tracking term.

Original source

This story was published by arXiv cs.LG and written by Denis Zorba, Michal Valko. SyncAI.news shows a preview; the complete article is on the publisher's site.

Read the full story on arxiv.org

Similar News