Improve Certified Training with Signal-to-Noise Ratio Loss to Decrease Neuron Variance and Increase Neuron Stability

TMLR Paper2276 Authors

Abstract:

This work addresses the issue of over-regularization in certified training, which often results in lower certified robustness.
By introducing the concepts of neuron variance and neuron stability, we delve into their roles in inducing over-regularization and affecting the model's certified robustness. To tackle the problem, we extend the Signal-to-Noise Ratio (SNR) into the realm of model robustness, offering a novel perspective and developing SNR-inspired losses aimed at optimizing neuron variance and stability to mitigate over-regularization. Through both empirical and theoretical analyses, our SNR-based approach demonstrates superior performance over existing methods on the MNIST and CIFAR-10 datasets. Further, our exploration into adversarial training uncovers a beneficial correlation between neuron variance and adversarial robustness, leading to an optimized balance between standard and robust accuracy that outperforms the baseline method.

Submission Length: Regular submission (no more than 12 pages of main content)

Assigned Action Editor: ~Gang_Niu1

Submission Number: 2276