Data Driven Regularization for Convolutional Neural Networks on Image Classification

Yihao Chen, Hanli Wang, Qinyu Li, Deshuang Huang

Abstract:

Deep Convolutional Neural Network has shown significant improvements in many fields of computer vision, and a series of researches are proposed to explore advanced model structures to attenuate the problem of over-fitting. In this paper, two data driven techniques are designed including SwitchNode and SwitchConnect, which employ the sparsity of deterministic data to regularize convolutional neural network models. Specifically, the proposed SwitchNode method switches from the redundant nodes which have similar activations and
spatial information to new initialization nodes, while the SwitchConnect method retrains replaceable convolutional kernels. The effectiveness of the proposed data driven regularization methods has been verified by the performance gain experimented on several benchmark image classification datasets.