
LG
Leonardo Galli, Daniela Angela Parletta, Andrea Paudice
· 1 min read
ResearcharXiv cs.LG
Taming the Tails of Stochastic Gradient Descent in the Interpolation Regime
arXiv:2611.24562v1 Announce Type: cross
Abstract: In this paper, we study stochastic gradient descent in the smooth interpolation regime, where the population objective shares a common minimizer with the sample losses. In contrast to prior work, which has largely focused on in-expectation analysis, we obtain high-probability convergence guarantees under the same assumptions. In the convex case, we prove a fast rate of order $\log(\beta^{-1}) \cdot n^{-1}$ after $n$ iterations, with failure probability at most $\beta$. In the strongly convex case, we derive a family of moment-based bounds of order $\beta^{-p} \cdot \exp(-\mathcal{O}(p \cdot n))$ for $p \in (0,1]$, and complement them with a matching lower bound showing that the corresponding deterioration in the exponential rate is essentially unavoidable. In both settings, our theoretical results are unified across a broad class of step-size schedules. Building on this analysis, we extend our results to a low-noise regime and to nonconvex objectives satisfying the Polyak-Lojasiewicz condition.
Original source
This story was published by arXiv cs.LG and written by Leonardo Galli, Daniela Angela Parletta, Andrea Paudice. SyncAI.news shows a preview; the complete article is on the publisher's site.
Read the full story on arxiv.org


