DP-ImgSyn: Dataset Alignment for Obfuscated, Differentially Private Image Synthesis
Abstract: The availability of abundant data has catalyzed the expansion of deep learning vision algorithms. However, certain vision datasets cannot be publicly released due to privacy reasons. Releasing synthetic images instead of private images is a common approach to overcome this issue. A popular method to generate synthetic images is using Generative Adversarial Networks (GANs) with Differential Privacy (DP) guarantees. However, GAN-generated synthetic images are visually similar to private images. This is a severe limitation, particularly when the private dataset depicts visually sensitive and disturbing content. To address this, we propose a non-generative framework, Differentially Private Image Synthesis (DP-ImgSyn), to generate and release synthetic images for image classification tasks. These synthetic images: (1) have DP guarantees, (2) retain the utility of the private images, i.e., a model trained using synthetic images results in similar accuracy as a model trained on private images, (3) the synthetic images are visually dissimilar to private images. DP-ImgSyn consists of the following steps: First, a teacher model is trained on the private images using a DP training algorithm. Second, public images are used as initialization for the synthetic images which are optimized to align them with the private images. The optimization uses the teacher network's batch normalization layer statistics (mean, standard deviation) to inject information about the private images into the synthetic images. Third, the synthetic images and their soft labels, obtained from the teacher model, are released and can be deployed for neural network training on image classification tasks. Our experiments on various image classification datasets show that when using similar DP training mechanisms, our framework performs better than generative techniques (up to $\approx$ 20% in terms of image classification accuracy).
Submission Length: Regular submission (no more than 12 pages of main content)
Previous TMLR Submission Url: https://openreview.net/forum?id=ZHAtvJtJnR&referrer=%5BAuthor%20Console%5D(%2Fgroup%3Fid%3DTMLR%2FAuthors%23your-submissions)
Changes Since Last Submission: Summary: We have made significant revisions to the introduction, added additional results and figures. We have improved the readability of the paper considering the feedback received from prior submission and review. We detail the specific changes below.
Regarding the feedback from the action editor:
* We corrected the typo related to the core of our work (public/private/synthetic images and similarity/dissimilarity). Specifically, in the previous version of the paper when describing the problem statement, it was written that the synthetic images must be **visually dissimilar to the public images**. The correct statement is that the synthetic images must be **visually dissimilar to the private images**. We fixed this typo, and we changed and redrawn Figure 1 to make it clear.
* We added a statement in Section 4.3 clarifying that our method needs the public images during the image synthesis process only. After the synthetic images are generated, they are released to train any network. Thus, the public images are not needed during the network training, as the network is trained using synthetic images.
* We revised the introduction to clarify the problem statement, i.e. we consider the problem of releasing synthetic images for image classification tasks that satisfy the following properties: (1) have (epsilon, delta$)-Differential Privacy (DP) guarantees, (2) Retain the utility of the private images, i.e., a model trained using synthetic images should result in similar classification accuracy as the model trained on private images, (3) are visually dissimilar to the private images. Visual dissimilarity is important for vision datasets that depict visually disturbing and sensitive content. Moreover, we specify in the revised introduction that our method falls in the category of data release methods and not model release methods and explain the advantages of data release methods over model release methods.
Regarding the feedback from the reviewers:
* We added more details in Section 4.2 about the selection of the target labels and added in Appendix A.1.3 the corresponding PyTorch code.
* We moved the image quality results (FID score) from the Appendix to Section 5.5 in the main paper so that it is easier for the reader to follow them, after the performance results.
* We explained the differences between semi-private learning and our work in Section 3.
* We revised Section 2 that to mention that the experimental results (Section 5) are in line with the intuition Figure 2. Moreover, we added in Appendix A.3 a section with visualizations of DP-ImgSyn using a real example. The video of the entire training process visualizing the decision boundaries and image synthesis (where we clearly see data movement) is included in the supplementary material.
* We added in Appendix A.1.2 the discussion for the hyperparameters that control the loss (Table 7).
* We added in Appendix A.5 the comparison with stronger privacy guarantees ($\epsilon$=0.2).
* We added in Appendix A.6 the requested experiments with low-pass filtering.
* We added the conceptual reason why l2 norm is needed in Section 4.2.
* We added a clarification about the reason of using the tv loss in Appendix A.2.
* We added a clarification in Section 3 to explain that using GAN generated images instead of public images for initializing the DP-ImgSyn would violate the visual dissimilarity property between the synthetic and the private images. The GAN generated images are visually similar to the private images, and thus the synthetic images initialized with GAN generated images would be visually similar to the private images. Thus, we use public images for the DP-ImgSyn initialization.
* We added the observation from Table 2 that the interference of the private and the public images is minimal in the main paper and point the reader to Appendix A.4 for more details.
Video: https://purdue0-my.sharepoint.com/:v:/g/personal/esoufler_purdue_edu/ER9699nkKH1ErxwbOMXDNVYBmmijTiFopGIfKf4MbQgtZg
Code: https://github.com/Efstathia-Soufleri/DP-ImgSyn
Supplementary Material: zip
Assigned Action Editor: ~Kamalika_Chaudhuri1
Submission Number: 2144
Loading