Keywords: information theory, NTK, ensemble, generalization
TL;DR: Infinite ensembles of infinitely wide neural networks are an interesting model family from an information theoretic perspective.
Abstract: In this preliminary work, we study the generalization properties of infinite ensembles of infinitely-wide neural networks. Amazingly, this model family admits tractable calculations for many information-theoretic quantities. We report analytical and empirical investigations in the search for signals that correlate with generalization.
Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/information-in-infinite-ensembles-of/code)
0 Replies
Loading