MeanSparse: Post-Training Robustness Enhancement Through Mean-Centered Feature Sparsification

Sajjad Amini, Mohammadreza Teymoorianfard, Shiqing Ma, Amir Houmansadr

Keywords: Adversarial Training, Sparsification, Robustness, Activation Functions, Proximal Operator

TL;DR: We enhance neural network robustness by reducing feature variation, setting new AutoAttack accuracy records on CIFAR-10, CIFAR-100, and ImageNet.

Abstract:

We present a simple yet effective method to improve the robustness of both Convolutional and attention-based Neural Networks against adversarial examples by post-processing an adversarially trained model.
Our technique, MeanSparse, cascades the activation functions of a trained model with novel operators that sparsify mean-centered feature vectors.
This is equivalent to reducing feature variations around the mean, and we show that such reduced variations merely affect the model's utility, yet they strongly attenuate the adversarial perturbations and decrease the attacker's success rate.
Our experiments show that, when applied to the top models in the RobustBench leaderboard, MeanSparse achieves a new robustness record of $75.28$% (from $73.71$%), $44.78$% (from $42.67$%) and $62.12$% (from $59.56$%) on CIFAR-10, CIFAR-100 and ImageNet, respectively, in terms of AutoAttack accuracy.
Code: https://anonymous.4open.science/r/MeanSparse-84B0/

Primary Area: alignment, fairness, safety, privacy, and societal considerations

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 11250