On the Difficulty of Feature Unlearning in Tabular Diffusion Models

Jeroen M. Galjaard, Chaoyi Zhu, Robert Birke, Pin-Yu Chen, Lydia Chen

Keywords: Generative Models, Diffusion Models, Structured Data, Tabular Data, Machine Unlearning, Feature Unlearning

TL;DR: Removing features from tabular diffusion models reveals a delicate balance between removing and preserving what matters.

Abstract:

Generative models have shown to be increasingly realistic data synthesizers.
Across various domains, this has urged for better control of the data that influence their behavior.
While unlearning methods show promise in image and language models, they remain understudied in the tabular context, generally, and in attribute-level (column/vertical) data removal, particularly.

Here, we aim to kick-off this investigation.
by proposing an evaluation framework to quantify the ability of post-training methods to remove features (columns).
Our evaluation of multiple unlearners and a new variant, Vertical Negative Preference Optimization, allow us to present insights into the dynamics of feature removal.
Therewith uncovering a consistent limitation: categorical features pose an additional challenge to unlearn successfully.

Email Sharing: We authorize the sharing of all author emails with Program Chairs.

Data Release: We authorize the release of our submission and author names to the public in the event of acceptance.

Submission Number: 73