Go to TMLR homepageReproducibility Study of "Improving Interpretation Faithfulness For Vision Transformers"
Abstract: This paper attempts to reproduce the findings of the study "Improving Interpretation Faith-fulness For Vision Transformers" Hu et al. (2024). The authors focus on making visual transformers (ViTs) more robust to adversarial attacks, and calling these robust ViTs faithful ViTs (FViTs). In their paper they propose a universal method to transform ViTs to FViTs called denoised diffusion smoothing (DDS). The reproduction of the authors study suffers from certain challenges, but the main claims still hold. Furthermore, this study extends the original paper by trying different diffusion models for DDS and tries to generalize the increased robustness of FViTs.
Certifications: Reproducibility Certification
Submission Length: Regular submission (no more than 12 pages of main content)
Changes Since Last Submission: - Following reviewer Q43y’s suggestion, we added the reference in the caption of figure 1 to the ViT paper.
- Following reviewer Q43y’s suggestion, we improved the structure and formatting of the appendix.
- Following reviewer Q43y’s, we rewrote the method section to make it clear that the perturbation is done on the pixel space.
- Following reviewer Q43y’s suggestion, we added the calculation for the optimal number of iterations as algorithm 4 in the appendix. We also rewrote the DDS pseudocode so that there are more details about how the optimal number of iterations is used.
- Following reviewer xjQS’ suggestion, we performed additional experiments using Kandinsky 2.2 and SDXL
- Following reviewer wFDK’s suggestion, we fixed the typographical error on page 6 in section 3.4
- Following reviewer wFDK’s suggestion, we performed additional experiments that compared the old aggregation method to the new one and added the results to the appendix.
Code: https://github.com/meherc99/FViTRepro
Supplementary Material: zip
Assigned Action Editor: ~Wei_Liu3
Submission Number: 4318
Loading