FUN: Filter-based Unlearnable Datasets

Vinu Sankar Sadasivan, Mahdi Soltanolkotabi, Soheil Feizi

Keywords: unlearnable examples, data protection, privacy, adversarial machine learning

TL;DR: We propose a novel, model-free convolutional filter-based unlearnable dataset (FUN) generation technique that protects data from empirical risk minimization and adversarial training with various budgets.

Abstract: Large-scale training of modern deep learning models heavily relies on publicly available data on the web. This potentially unauthorized usage of online data leads to concerns regarding data privacy. Recent works aim to make unlearnable data for deep learning models by adding small, specially designed noises to tackle this issue. However, these methods are vulnerable to adversarial training (AT) and/or are computationally heavy. In this work, we propose a novel, model-free convolutional Filter-based UNlearnable (FUN) dataset generation technique. FUN performs controlled class-wise convolutions using filters that are randomly generated via a private key. FUN encourages the network to learn the relation between filters and labels rather than informative features for classifying the clean data. We develop some theoretical analysis demonstrating that FUN can successfully poison Gaussian mixture data by reducing the clean data performance of the optimal Bayes classifier. We also empirically demonstrate the effectiveness of FUN with various datasets (CIFAR-10, CIFAR-100, and ImageNet-100), and architectures (ResNet-18, VGG-16, Wide ResNet-34-10, and DenseNet-121). Our experiments show that FUN is robust to various data augmentations and training approaches such as smoothing, AT with different budgets, transfer learning, and fine-tuning. For instance, training a ResNet-18 on FUN ImageNet-100 data achieves only 8.96$\%$, 40.08$\%$, and 20.58$\%$ clean test accuracies with empirical risk minimization (ERM), $L_{\infty}$ AT, and $L_{2}$ AT, respectively. Here, ERM on the clean training data achieves a clean test accuracy of 80.66$\%$. Furthermore, we also show that FUN is robust to adaptive defenses designed specifically to break it.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Submission Guidelines: Yes

Please Choose The Closest Area That Your Submission Falls Into: Applications (eg, speech processing, computer vision, NLP)