Knowledge Editing through Chain-of-Thought

ACL ARR 2025 February Submission1610 Authors

Abstract:

Knowledge Editing is a technique that updates large language models (LLMs) with new information to maintain their world knowledge.
This approach avoids the need to rebuild the model from scratch, thereby addressing the high costs associated with frequent retraining.
Among these, the in-context editing paradigm stands out for its effectiveness in integrating new knowledge while preserving the model's original capabilities.
Despite its potential, existing in-context knowledge editing methods are often task-specific, focusing primarily on multi-hop QA tasks using structured knowledge triples.
Moreover, their reliance on few-shot prompting for task decomposition makes them unstable and less effective in generalizing across diverse tasks.
In response to these limitations, we propose EditCoT, a novel knowledge editing framework that flexibly and efficiently updates LLMs across various tasks without retraining.
EditCoT works by generating a chain-of-thought (CoT) for a given input and then iteratively refining this CoT process using a CoT editor based on updated knowledge.
We evaluate EditCoT across a diverse range of benchmarks, covering multiple languages and tasks.
The results demonstrate that our approach achieves state-of-the-art performance while offering superior generalization, effectiveness, and stability compared to existing methods, marking a significant advancement in the field of knowledge updating.

Paper Type: Long

Research Area: Generation

Research Area Keywords: knowledge-augmented methods

Contribution Types: NLP engineering experiment, Approaches to low-resource settings

Languages Studied: English, Chinese

Submission Number: 1610