Beyond Sample-Level Forgetting: Improving Reliability in Multimodal Unlearning

Jianzhou Wang, Yirui Wu, Lixin Yuan, Wenxiao Zhang, Jun Liu

Abstract:

Multimodal unlearning aims to eliminate specific data from pretrained multimodal models, which offers significant advantages in data privacy and model efficiency.
Current methods struggle to achieve the desired properties of effectiveness, reliability and locality, due to the complex interdependency of unimodal and multimodal knowledge. By introducing a causal perspective, we propose multimodal unlearning with decoupled knowledge components. To promote fine-grained understanding of multimodal context, we introduce Multimodal Variational Inference (MVI) to infer modal-specific and -consistent factors with incomplete sample observation.
With foundation of decoupled knowledge, we propose contrastive semantic editing to regulate multimodal unlearning towards refined forgetting. Experiments on privacy- and copyright-sensitive scenarios validate effectiveness of our method across multiple scenarios, ensuring the unlearned model maintains high reliability and locality.

Lay Summary:

Modern Artificial Intelligence (AI) models learn about the world by combining different types of data, such as text and images. But what happens if an AI accidentally learns copyrighted artwork or private personal data? We need a way to make the AI "forget" that specific information—a process known as machine unlearning.

Currently, this is incredibly difficult because text and image memories are deeply tangled together inside the model. Trying to erase one specific piece of data often accidentally damages unrelated knowledge, making the AI less reliable.

To solve this, we introduced a cause-and-effect framework to safely untangle the AI’s memory. Our method splits the model's knowledge into independent pieces—separating what it learned from text, what it learned from images, and what they share. This allows us to precisely edit and remove only the unwanted data.

We tested our approach on privacy- and copyright-sensitive scenarios. The results show that our method successfully deletes the targeted data while ensuring the AI remains completely stable, accurate, and helpful on all other tasks.

Primary Area: Social Aspects->Privacy

Keywords: multimodal learning, machine unlearning, variational inference, causal inference

Submission Number: 32509