AlphaEdit+: Model Editing in the Presence of Conflicting and Inconsistent Knowledge

ICLR 2026 Conference Submission216 Authors

Keywords: Knowledge Editing; Null-space Perturbation; Knowledge Conflict; Knowledge Inconsistency

TL;DR: We propose AlphaEdit+, a framework that resolves knowledge conflicts and inconsistencies in model editing through a null-space perturbation, conflict-aware weighting and objective smoothing.

Abstract:

Knowledge editing is a crucial technique for daily updates in LLMs, requiring a balance between accurately modifying incorrect knowledge and preserving existing information. The recently proposed AlphaEdit method achieves competitive editing performance by updating parameters under null-space constraints. However, our theoretical analysis reveals that AlphaEdit struggles with high knowledge conflicts
and inconsistencies during editing. To address this, we propose a new editing method AlphaEdit+, featuring three key improvements: 1) relaxing null-space constraints by adding a matrix perturbation through optimization to resolve conflicts between new and preserved knowledge; 2) introducing a weighting scheme on previously updated knowledge constraints to mitigate conflicts between new and
historical editing; 3) developing a value smoothing algorithm to resolve high knowledge inconsistencies. These enhancements collectively ensure robust editing while maintaining model coherence. Comprehensive experiments show that our approach AlphaEdit+ not only resolves the brittleness of the original method on carefully constructed challenging datasets but also achieves slightly better performance than AlphaEdit on existing benchmark datasets.

Primary Area: foundation or frontier models, including LLMs

Submission Number: 216