MACROBERT: Maximizing Certified Region of BERT to Adversarial Word Substitutions

Fali Wang, Zheng Lin, Zhengxiao Liu, Mingyu Zheng, Lei Wang, Daren Zha

Abstract: Deep neural networks are deemed to be powerful but vulnerable, because they will be easily fooled by carefully-crafted adversarial examples. Therefore, it is of great importance to develop models with certified robustness, which can provably guarantee that the prediction will not be easily misled by any possible attack. Recently, although a certified method based on randomized smoothing is proposed, it does not take the maximized certified region into account, so we develop an approach to train models with maximized certified regions via replacing the base classifier with the soft smoothed classifier which is differentiable during propagation.