SyncAI.news, a Varaisys broadcasting
Rescaling OLS-Merged Adapters To Improve Personalized Image Generation
LG

Leander Girrbach, Niklas Schweiger, Zeynep Akata

· 1 min read

ResearcharXiv cs.CV

Rescaling OLS-Merged Adapters To Improve Personalized Image Generation

arXiv:2611.22969v1 Announce Type: new Abstract: Merging multiple adapters into a single image generation model reduces deployment costs while preserving diverse task capabilities or personalization modes. State-of-the-art methods such as RegMean, Mix-of-Show, and Subspace Routing fit the outputs of the merged model to those of the individual adapters using ordinary least squares (OLS). We show that this reconstruction objective systematically shrinks task-specific signals whenever the activation distributions of different tasks overlap, producing a merged adapter that is too weak for downstream use. We derive a decomposition that quantifies this shrinkage and propose rescaling the merged update to restore signal strength. On multi-subject personalization with up to 30 merged adapters, rescaling substantially improves subject fidelity over unscaled baselines. Our approach also improves results on style transfer, concept erasure, image classification, and LLM-based text classification.

Original source

This story was published by arXiv cs.CV and written by Leander Girrbach, Niklas Schweiger, Zeynep Akata. SyncAI.news shows a preview; the complete article is on the publisher's site.

Read the full story on arxiv.org

Similar News