On Conformal Machine Unlearning

Yahya Alkhatib, Wee Peng Tay

Keywords: Machine Unlearning, Conformal Prediction

TL;DR: This paper lays the foundation for a new definition of machine unlearning based on the uncertainty-quantification scheme of conformal prediction.

Abstract: The increasing demand for data privacy has made Machine Unlearning (MU) essential for removing the influence of specific training samples from machine learning models while preserving performance on retained data. However, most existing MU methods lack rigorous statistical guarantees or rely on heuristic metrics such as accuracy. To overcome these limitations, we introduce a new definition for MU based on Conformal Prediction (CP), providing statistically sound, uncertainty-aware guarantees without the need for the concept of naive retraining. We formalize the proposed conformal criteria that quantify how often forgotten samples are excluded from CP sets, and propose empirical metrics—the Efficiently Covered Frequency (ECF at $c$) and its complement, the Efficiently Uncovered Frequency (EuCF at $d$)—to measure the effectiveness of unlearning. We further present a practical unlearning method designed to optimize these conformal metrics. Extensive experiments across diverse forgetting scenarios, datasets and models demonstrate the efficacy of our approach in removing targeted data.

Primary Area: alignment, fairness, safety, privacy, and societal considerations

Submission Number: 11818