Patient-level Machine Unlearning in Latent Diffusion Models: On the Limits of the Privacy-Utility Trade-off

Inês Cardoso, Tiago Filipe Sousa Gonçalves, Luis Filipe Teixeira, Wilson Silva

Keywords: case-based explanations, federated learning, machine unlearning, privacy

Registration Requirement: Yes

Abstract: While federated learning frequently uses synthetic images for case-based explanations, diffusion models pose a privacy risk by potentially memorizing and recreating patient-identifiable data. Machine unlearning offers a way to remove specific training data influences, but its effectiveness for patient-level anonymization in generative models is not yet well understood. In this work, we present the first empirical analysis of patient-level unlearning in latent diffusion models, testing three strategies, including our novel KL-Away approach. Our results reveal a critical trade-off: methods that successfully unlearn data degrade diagnostic utility, whereas utility-preserving techniques fail to protect privacy, leaving over 20% of patients re-identifiable. We attribute this to feature entanglement and distributed memorization, suggesting that existing unlearning techniques are currently insufficient for reliable patient anonymization.

Visa & Travel: No

Read CFP & Author Instructions: Yes

Originality Policy: Yes

Single-blind & Not Under Review Elsewhere: Yes

LLM Policy: Yes

Submission Number: 100