SyncAI.news, a Varaisys broadcasting
CT-Merging: Consensus Directions and Task-Specific Scaling for LoRA Adapter Merging
KR

Keumseo Ryum, Joonhyuk Kang

· 1 min read

ResearcharXiv cs.CV

CT-Merging: Consensus Directions and Task-Specific Scaling for LoRA Adapter Merging

arXiv:2607.20561v2 Announce Type: replace-cross Abstract: LoRA merging methods increasingly operate on the low-rank structure of task updates, yet how the common subspace is estimated and how coefficients are assigned after recomposition are rarely compared directly. We propose CT-Merging, which estimates common directions from averaged task subspace projectors and assigns a separate residual scale to each task. Projector averaging selects directions supported across task subspaces without weighting them by singular magnitude, while task-specific scaling removes component-wise magnitude variation and preserves scale differences across tasks. On the released KnOTS CLIP adapters, CT-Merging achieves the best average and worst-task normalized accuracy on both backbones, improving over the strongest baseline by up to 2.56 and 6.65 points, respectively. On the DC-Merge adapter benchmark, it achieves the best average normalized accuracy in eight of nine backbone and task-count settings. Ablations show that projector averaging outperforms summed-update SVD and that task-specific scaling improves worst-task accuracy over global isotropic scaling.

Original source

This story was published by arXiv cs.CV and written by Keumseo Ryum, Joonhyuk Kang. SyncAI.news shows a preview; the complete article is on the publisher's site.

Read the full story on arxiv.org

Similar News