Towards Robust Detection and Segmentation Using Vertical and Horizontal Adversarial Training

Yongduo Sui, Tianlong Chen, Pengfei Xia, Shuyao Wang, Bin Li

Abstract: Adversarial training (AT) commonly serves as an advanced regularization to establish enhanced robust models. However, it usually scarifies performance on clean inputs, especially in complicated object detection and semantic segmentation tasks. However, how to fully unleash the power of adversarial training regularization to improve the trade-off between standard performance and adversarial robustness of detection and segmentation models, has not been explored. In this paper, we present the Vertical and Horizontal Adversarial Training (VHAT) regularization on both input and intermediate features, which consists of two major components: i) Vertical Adversarial Training (VAT) by utilizing adversarial features with a wide range of attack strengths; ii) Horizontal Adversarial Training (HAT) by injecting layer-wise adversarial feature perturbations together with adversarial samples. Extensive experiment results demonstrate that VHAT achieves the standard performance and adversarial robustness double-win for Faster-RCNN on PASCAL VOC and DeepLabv3+ on PASCAL VOC and Cityscapes datasets, respectively. Comprehensive ablation studies and visualizations are provided to reveal the insights and working mechanisms.