
FM
Federico Milanesio, Alessandro Ingrosso, Matteo Osella
· 1 min read
ResearcharXiv cs.LG
An Analytical Theory of Auxiliary Learning
arXiv:2609.29774v1 Announce Type: new
Abstract: Auxiliary learning is an optimization paradigm in which a neural network's performance on a target task is improved by jointly training it on additional tasks. However, the mechanisms behind this improvement remain poorly understood. We study this problem using a teacher-student framework and derive a closed system of differential equations describing the dynamics of online stochastic gradient descent in the large-input limit. For linear networks, we obtain a closed-form expression for the generalization error to leading order in the learning rate, quantifying how task correlations and label noise determine the benefit of auxiliary learning. For non-linear activation functions, we develop a fluctuation-dissipation analytical theory that establishes a general relation linking the main and auxiliary errors to the corresponding single-task error. Numerical experiments support the theoretical predictions and show how auxiliary tasks improve generalization by balancing the forcing dynamics towards the optimal solution with gradient noise.
Original source
This story was published by arXiv cs.LG and written by Federico Milanesio, Alessandro Ingrosso, Matteo Osella. SyncAI.news shows a preview; the complete article is on the publisher's site.
Read the full story on arxiv.org


