Robust LLM Unlearning with MUDMAN: Meta-Unlearning with Disruption Masking And Normalization

Filip Sondej, Yushi Yang, Mikolaj Kniejski, Marcel Windys

Keywords: unlearning, representation-engineering, language-models, biosecurity, fine-tuning, robustness, adversarial-attacks, WMDP, AI-safety, neural-representations, evaluation-robustness, meta-learning, MAML, machine-unlearning, model-editing

TL;DR: LLM unlearning is more robust if we only allow updating weights where the unlearning and retaining gradients have the same sign.

Abstract:

Language models can retain dangerous knowledge and skills even after extensive safety fine-tuning, posing both misuse and misalignment risks. Recent studies show that even specialized unlearning methods can be easily reversed. To address this, we systematically evaluate existing ones and propose novel components of unlearning methods and identify ones crucial for irreversible unlearning.

We introduce Disruption Masking, a technique in which we only allow updating weights, where the signs of the unlearning gradient and the retaining gradient are the same. This ensures all updates are non-disruptive.

Additionally, we identify the need for normalizing the unlearning gradients, and also confirm the usefulness of meta-learning. We combine these insights into MUDMAN (Meta-Unlearning with Disruption Masking and Normalization) and validate its effectiveness at preventing the recovery of dangerous capabilities. MUDMAN outperforms the prior TAR method by 40%, setting a new state-of-the-art for robust unlearning.

Serve As Reviewer: ~Yushi_Yang2

Submission Number: 15