Editing the Mind of Giants: An In-Depth Exploration of Pitfalls of Knowledge Editing in Large Language Models
Abstract: Knowledge editing is a rising technique for efficiently updating factual knowledge in large language models (LLMs) with minimal alteration of parameters. However, recent studies have identified concerning side effects, such as knowledge distortion and the deterioration of general abilities, that have emerged after editing. This paper conducts a comprehensive study of these side effects, providing a unified view of the challenges associated with knowledge editing in LLMs. We discuss related work and summarize potential research directions to overcome these limitations. Our experiments highlight the limitations of current knowledge editing methods, emphasizing the need for deeper understanding of inner knowledge structures of LLMs and improved knowledge editing methods.
Paper Type: Long
Research Area: Machine Learning for NLP
Research Area Keywords: knowledge editing, large language model
Contribution Types: Surveys
Languages Studied: English
Submission Number: 685
Loading