Machine Unlearning in Audio: Bridging The Modality Gap Via the Prune and Regrow Paradigm

ICLR 2025 Conference Submission13858 Authors

28 Sept 2024 (modified: 27 Nov 2024)ICLR 2025 Conference SubmissionEveryoneRevisionsBibTeXCC BY 4.0
Keywords: Machine Unlearning, Audio, Deep Learning, Privacy, Pruning
TL;DR: Addressing the audio modality gap in machine unlearning literature.
Abstract: The ubiquity and success of deep learning is primarily owed to large human datasets; however, increasing interest in personal data raises questions of how to satisfy privacy legislation in deep learning. Machine unlearning is a nascent discipline centred on satisfying user privacy demands, by enabling data removal requests on trained models. While machine unlearning has reached a good level of maturity in the vision and language domains, applications in audio are largely underexplored, despite it being a highly prevalent and widely used modality. We address this modality gap by providing the first systematic analysis of machine unlearning techniques covering multiple architectures trained on audio datasets. Our analysis highlights that in audio, existing methods fail to remove data for the most likely case of unlearning -- Item Removal. We present a novel Prune and Regrow Paradigm that bolsters sparsity unlearning through Cosine and Post Optimal Pruning, achieving the best unlearning accuracy for 9/12 (75%) of Item Removal experiments and best, or joint best, for for 50% (6/12) of Class Removal Experiments. Furthermore, we run experiments showing performance as unlearning requests scale, and we shed light on the mechanisms underpinning the success of our Prune and Regrow Paradigm.
Primary Area: alignment, fairness, safety, privacy, and societal considerations
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Reciprocal Reviewing: I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 13858
Loading