Privacy-Preserving Split Learning with Vision Transformers using Patch-Wise Random and Noisy CutMix
Abstract: In computer vision, the vision transformer (ViT) has increasingly superseded the convolutional neural network (CNN) for improved accuracy and robustness. However, ViT's large model sizes and high sample complexity make it difficult to train on resource-constrained edge devices. Split learning (SL) emerges as a viable solution, leveraging server-side resources to train ViTs while utilizing private data from distributed devices. However, SL requires additional information exchange for weight updates between the device and the server, which can be exposed to various attacks on private training data. To mitigate the risk of data breaches in classification tasks, inspired from the CutMix regularization, we propose a novel privacy-preserving SL framework that injects Gaussian noise into smashed data and mixes randomly chosen patches of smashed data across clients, coined DP-CutMixSL. Our analysis demonstrates that DP-CutMixSL is a differentially private (DP) mechanism that strengthens privacy protection against membership inference attacks during forward propagation. Through simulations, we show that DP-CutMixSL improves privacy protection against membership inference attacks, reconstruction attacks, and label inference attacks, while also improving accuracy compared to DP-SL and DP-MixSL.
Submission Length: Long submission (more than 12 pages of main content)
Previous TMLR Submission Url: https://openreview.net/forum?id=E4mUkIJ9kn&referrer=%5BAuthor%20Console%5D
Changes Since Last Submission: 1. Regarding Scope:
As per reviewer wz2M's suggestion, we reviewed existing work on privacy-preserving SL in ViT. We have clearly added the scope differences from our work to the related works and conclusions. Additionally, as per reviewer eXdQ's comments, we have included a description and initial results of DP-CutMixSL's potential challenges with patch shuffling in the conclusions and Appendix G, respectively.
2. Regarding Technical Details:
In response to comments from reviewers eXdQ and wz2M, we have added more details on the mixer and noise, discussed the dropout layer, and provided settings for the auxiliary network for the reconstruction attack.
3. Regarding Presentation:
In response to reviewers wz2M and Zgri, we have revised the abstract and introduction, reducing unnecessary details and clearly defining terms, as well as complementing related works.
4. Regarding Experiments:
As per reviewer Zgri's suggestion, we updated the plotting of Figure 5a. Additionally, we added the attack success rate for membership inference attacks and visualized reconstruction attacks in the main text, as per reviewer wz2M's comment.
Assigned Action Editor: ~Kamalika_Chaudhuri1
Submission Number: 2608
Loading