Regularization of Neural Networks using DropConnect

Li Wan, Matthew D. Zeiler, Sixin Zhang, Yann LeCun, Rob Fergus

Abstract: We introduce DropConnect, a generalization of DropOut, for regularizing large fully-connected layers within neural networks. When training with Dropout, a randomly selected subset of activations ar...