Keywords: information theory, NTK, ensemble, generalization
TL;DR: Infinite ensembles of infinitely wide neural networks are an interesting model family from an information theoretic perspective.
Abstract: In this preliminary work, we study the generalization properties of infinite ensembles of infinitely-wide neural networks. Amazingly, this model family admits tractable calculations for many information-theoretic quantities. We report analytical and empirical investigations in the search for signals that correlate with generalization.
Community Implementations: [![CatalyzeX](/images/catalyzex_icon.svg) 1 code implementation](https://www.catalyzex.com/paper/arxiv:1911.09189/code)
0 Replies
Loading