SyncAI.news, a Varaisys broadcasting
ManiEdit: Sequential Unstructured Knowledge Editing for Language Models from a Manifold Perspective
RL

Rui Liu, Chenheng Zhang, Haoxuan Li, Zhouchen Lin

· 1 min read

ResearcharXiv cs.CL

ManiEdit: Sequential Unstructured Knowledge Editing for Language Models from a Manifold Perspective

arXiv:2609.33534v1 Announce Type: new Abstract: Large language models (LLMs) inevitably generate some incorrect or outdated content, necessitating efficient and precise mechanisms for continual knowledge updates. However, existing model editing methods struggle to sequentially edit unstructured long-form knowledge, suffering from severe edit forgetting and degradation of general capabilities. To address these challenges, we reframe knowledge editing from a manifold perspective, viewing it as a localized displacement of an edit sub-manifold within the global knowledge manifold. Under this formulation, the problem can be decomposed into two key questions: (i) how to identify representative edit points that effectively anchor the edit sub-manifold, and (ii) how to preserve the remaining manifold structure during the sub-manifold displacement process. Based on this perspective, we propose ManiEdit, a novel manifold-aware autoregressive editing framework consisting of two core components. Pivot Localization addresses the mediocre-point dilemma by identifying high-leverage pivots to anchor the edit sub-manifold. Manifold-Aware Preservation preserves different knowledge types through an energy-weighted penalty combined with recursive null-space alignment. Experiments on two base LLMs and four unstructured editing benchmarks demonstrate that ManiEdit achieves state-of-the-art performance, outperforming the strongest baseline by up to +27.81 BERTScore and +8.50 ROUGE-L, while maintaining near-original general capabilities across six representative downstream tasks. Our code is available at: https://github.com/Areyliu/ManiEdit

Original source

This story was published by arXiv cs.CL and written by Rui Liu, Chenheng Zhang, Haoxuan Li, Zhouchen Lin. SyncAI.news shows a preview; the complete article is on the publisher's site.

Read the full story on arxiv.org

Similar News