Towards Understanding the Effect of NTP Paradigm in Unstructured Knowledge Editing

Zisheng Zhou, Mengqi Zhang, Shiguang Wu, Xiaotian Ye, Chi Zhang, Zhumin Chen, Pengjie Ren

Keywords: Large language models, Knowledge editing

TL;DR: We identify the context reliance problem in the NTP paradigm for unstructured knowledge editing and develop a simple yet effective mitigation strategy.

Abstract:

Editing Large language models (LLMs) with real-world, unstructured knowledge is critical for correcting and updating their internal knowledge bases. However, current methods often oversimplify this knowledge, leading to information loss and suboptimal performance.
While existing editing techniques based on the next-token prediction (NTP) paradigm show promise, our investigation reveal a core limitation: context reliance. The edited knowledge heavily rely on the preceding context available during editing, but this context is often absent in practical inference. This gap between editing and inference limits the generalization of acquired knowledge. We validate this issue both theoretically and experimentally, demonstrating that the absence of preceding context prevents model from recalling the edited knowledge, thereby causing a performance drop on editing success rate.
To address this, we propose a simple yet effective COntext-INdependent unstructured knowledge editing framework (COIN), encouraging the model to internalize new knowledge properly, rather than merely memorizing fixed patterns with its preceding context.
Comprehensive evaluations show that COIN significantly reduces the performance drop and outperforms strong baselines by 23.6\% in editing success rate, highlighting the potential of NTP paradigm for robust unstructured knowledge editing.

Primary Area: foundation or frontier models, including LLMs

Submission Number: 3922