Position: Lifting Data for Foundation Model Unlearning

Yuwen Tan, Boqing Gong

TL;DR: Positions on Machine Unlearning for Foundation Models.

Abstract: Machine unlearning removes certain training data points and their influence on AI models (e.g. when a data owner revokes their decision to allow models to learn from the data). In this position paper, we propose to lift data-tracing machine unlearning to knowledge-tracing for foundation models (FMs). We support this position based on practical needs and insights from cognitive studies. Practically, tracing data cannot meet the diverse unlearning requests for FMs, which may be from regulators, enterprise users, product teams, etc., having no access to FMs' massive training data. Instead, it is convenient for these parties to issue an unlearning request about the knowledge or capability FMs (should not) possess. Cognitively, knowledge-tracing unlearning aligns with how the human brain forgets more closely than tracing individual training data points. Finally, we provide a case study using a vision-language foundation model to deepen the discussions.

Primary Area: Research Priorities, Methodology, and Evaluation

Keywords: Machine Unlearning, Foundation Model, Human Forgetting

Submission Number: 162