Penalizing Gradient Norm for Efficiently Improving Generalization in Deep Learning

Yang Zhao, Hao Zhang, Xiuyuan Hu

Abstract: How to train deep neural networks (DNNs) to generalize well is a central concern in deep learning, especially for severely overparameterized networks nowadays. In this paper, we propose an effectiv...