Resolving UnderEdit & OverEdit with Iterative & Neighbor-Assisted Model Editing

ACL ARR 2025 May Submission5102 Authors

Abstract: Large Language Models (LLMs) are widely deployed in downstream tasks, but keeping their knowledge up-to-date via retraining or fine-tuning is often computationally expensive. Model editing provides a more efficient alternative by updating a targeted subset of parameters, which often follows the locate-and-edit paradigm. Despite this efficiency, existing methods are limited: edits may fail to inject knowledge (UnderEdit) or unintentionally disrupt unrelated neighboring knowledge (OverEdit). To address these challenges, we propose two complementary methods: $\textbf{iterative model editing}$, which applies successive edits to mitigate UnderEdit, and $\textbf{neighbor-assisted model editing}$, which incorporates neighboring knowledge during editing to reduce OverEdit. Our extensive experiments show that these techniques improve editing performance across multiple LLMs, algorithms, and benchmarks, reducing UnderEdit by up to 38 percentage points and OverEdit by up to 6, while remaining broadly applicable to any locate-and-edit method.

Paper Type: Long

Research Area: Machine Learning for NLP

Research Area Keywords: Model Editing, Optimization

Contribution Types: Model analysis & interpretability

Languages Studied: English

Submission Number: 5102