CURaTE: Continual Unlearning in Real Time with Ensured Preservation of LLM Knowledge

ACL ARR 2026 January Submission1678 Authors

Keywords: Unlearning, real time unlearning, continual unlearning, large language model

Abstract: The inability to filter out in advance all potentially problematic data from the pre-training of large language models has given rise to the need for methods for unlearning specific pieces of knowledge after training. Existing techniques overlook the need for continuous and immediate action, causing them to suffer from degraded utility as updates accumulate and protracted exposure of sensitive information. To address these issues, we propose **C**ontinual **U**nlearning in **R**e**a**l **T**ime with **E**nsured Preservation of LLM Knowledge (**CURaTE**). Our method begins by training a sentence embedding model on a dataset designed to enable the formation of sharp decision boundaries for determining whether a given input prompt corresponds to any stored forget requests. The similarity of a given input to the forget requests is then used to determine whether to answer or return a refusal response. We show that even with such a simple approach, not only does **CURaTE** achieve more effective forgetting than existing methods, but by avoiding modification of the language model parameters, it also maintains near perfect knowledge preservation over any number of updates and is the only method capable of continual unlearning in real-time.

Paper Type: Long

Research Area: Safety and Alignment in LLMs

Research Area Keywords: Ethics, Bias, and Fairness, Language Modeling

Contribution Types: NLP engineering experiment, Approaches low compute settings-efficiency

Languages Studied: English

Submission Number: 1678