CoUn: Empowering Machine Unlearning via Contrastive Learning

Yasser H. Khalil, Mehdi Setayesh, Hongliang Li

Keywords: Machine Unlearning, Contrastive Learning, Supervised Learning, Representation Learning, Semantic Similarity

TL;DR: This paper presents CoUn, a contrastive learning (CL)-based machine unlearning (MU) framework using only retain data. Further, our proposed CL module can be integrated with existing baselines to empower their performance.

Abstract: Machine unlearning (MU) aims to remove the influence of specific ''forget'' data from a trained model while preserving its knowledge of the remaining ''retain'' data. Existing MU methods based on label manipulation or model weight perturbations often achieve limited unlearning effectiveness. To address this, we introduce CoUn, a novel MU framework inspired by the observation that a model retrained from scratch using only retain data classifies forget data based on their semantic similarity to the retain data. CoUn emulates this behavior by adjusting learned data representations through contrastive learning (CL) and supervised learning, applied exclusively to retain data. Specifically, CoUn (1) leverages semantic similarity between data samples to indirectly adjust forget representations using CL, and (2) maintains retain representations within their respective clusters through supervised learning. Extensive experiments across various datasets and model architectures show that CoUn consistently outperforms state-of-the-art MU baselines in unlearning effectiveness. Additionally, integrating our CL module into existing baselines empowers their unlearning effectiveness.

Primary Area: General machine learning (supervised, unsupervised, online, active, etc.)

Submission Number: 7407