SyncAI.news, a Varaisys broadcasting
Nearly Tight Almost Sure Convergence Rates for the Last iterate of SGD
WO

Weiming Ou, Andrea Paudice, Xiao Wang

· 1 min read

ResearcharXiv cs.LG

Nearly Tight Almost Sure Convergence Rates for the Last iterate of SGD

arXiv:2611.24151v1 Announce Type: cross Abstract: We study the almost sure convergence of the last-iterate optimization error of Stochastic (sub)Gradient Descent (SGD) for convex objectives. For smooth objectives, we establish convergence rates of order $k^{-1/3}$ when a stationary minimizer exists, and $k^{-1/4}$ otherwise, up to polylogarithmic factors, where $k$ denotes the iteration counter. We further show that the latter rate extends to nonsmooth convex objectives. All these rates are achieved by deterministic, anytime stepsize policies. Finally, we establish matching lower bounds showing that the exponents $-1/3$ and $-1/4$ are optimal, up to polylogarithmic factors, for the corresponding problem classes.

Original source

This story was published by arXiv cs.LG and written by Weiming Ou, Andrea Paudice, Xiao Wang. SyncAI.news shows a preview; the complete article is on the publisher's site.

Read the full story on arxiv.org

Similar News