Information in Infinite Ensembles of Infinitely-Wide Neural NetworksDownload PDF

16 Oct 2019 (modified: 06 Dec 2019)AABI 2019 Symposium Blind SubmissionReaders: Everyone
  • Keywords: information theory, NTK, ensemble, generalization
  • TL;DR: Infinite ensembles of infinitely wide neural networks are an interesting model family from an information theoretic perspective.
  • Abstract: In this preliminary work, we study the generalization properties of infinite ensembles of infinitely-wide neural networks. Amazingly, this model family admits tractable calculations for many information-theoretic quantities. We report analytical and empirical investigations in the search for signals that correlate with generalization.
0 Replies