
IK
Ivan Karpukhin, Andrey Savchenko
· 1 min read
ResearcharXiv cs.LG
AlignedHPO: Efficient Surrogate-Guided Loss Hyperparameter Optimization
arXiv:2611.22658v1 Announce Type: new
Abstract: Modern deep learning losses involve multiple hyperparameters, such as weights, scales, and margins, typically tuned manually or through expensive black-box optimization despite their differentiability. We propose AlignedHPO, an efficient framework for tuning differentiable loss hyperparameters using gradient alignment and surrogate validation objectives. Our method employs Jacobian-vector products to reduce hypergradient computation time and memory, while differentiable metric surrogates guide optimization toward task-specific evaluation criteria. We address gradient magnitude ambiguity and stochastic instability through normalized alignment and gradient smoothing. Experiments with generalized Focal Loss and ArcFace demonstrate effective adaptation across long-tailed classification and open-set face recognition. AlignedHPO achieves competitive or improved performance compared to implicit differentiation, with nearly 12x faster hypergradient computation. These results highlight the potential of efficient surrogate-guided loss hyperparameter optimization.
Original source
This story was published by arXiv cs.LG and written by Ivan Karpukhin, Andrey Savchenko. SyncAI.news shows a preview; the complete article is on the publisher's site.
Read the full story on arxiv.org


