MuEdit: A Lightweight yet Effective Multi-task Model Editing Method

ICLR 2026 Conference Submission4779 Authors

Keywords: Multi-task model editing, Model editing, Null-space

TL;DR: We introduce a novel multi-task learning paradigm.

Abstract: Large language models (LLMs) are prone to misinterpreting instructions and generating incorrect responses, stimulating the development of model editing methods. Existing model editing methods, however, face limitations in handling multi-task knowledge updates due to interference between different tasks. This paper firstly by analyzing the shortcomings of traditional editing methods and Fang's null-space projection method, which fails to generalize to multi-task scenarios because the null-space projection matrix of the current task may not lie within that of previous tasks, causing conflicts. To tackle this, we propose a new concept, the \textbf{Conflict Index}, to measure conflicts between two tasks' editing objectives. We then design two methods: \textbf{finding the optimal editing path} to minimize the total conflict index and \textbf{using a low-rank matrix expression method based on the conflict index to expand the null-space dimension} when conflicts remain high. Experimental results show that our proposed Mu-Edit method effectively alleviates multi-task editing conflicts, outperforming existing baseline methods in various metrics across multiple tasks and able to preserve abilities in general domains.

Primary Area: interpretability and explainable AI

Submission Number: 4779