A Random Matrix Analysis of Learning with α-DropoutDownload PDF

May 16, 2020 (edited Jul 09, 2020)ICML 2020 Workshop Artemiss SubmissionReaders: Everyone
  • Keywords: random matrix theory, dropout, zero imputation
  • Abstract: This article studies a single hidden layer neural network with generalized Dropout (α-Dropout), where the dropped out features are replaced with an arbitrary value α. Specifically, under a large dimensional data and network regime, we provide the generalization performances for this network on a binary classification problem. We notably demonstrate that a careful choice of α different from 0 can drastically improve the generalization performances of the classifier.
2 Replies