- Keywords: information theory, NTK, ensemble, generalization
- TL;DR: Infinite ensembles of infinitely wide neural networks are an interesting model family from an information theoretic perspective.
- Abstract: In this preliminary work, we study the generalization properties of infinite ensembles of infinitely-wide neural networks. Amazingly, this model family admits tractable calculations for many information-theoretic quantities. We report analytical and empirical investigations in the search for signals that correlate with generalization.