Neural Polytopes

Published: 28 Jul 2023, Last Modified: 28 Jul 2023SynS & ML @ ICML2023EveryoneRevisionsBibTeX
Keywords: polytopes, linear activation, discrete geometry
TL;DR: We find that various polytopes are neural networks, and generalize polytopes, to bridge machine learning and discrete geometry.
Abstract: We find that simple neural networks with ReLU activation generate polytopes as an approximation of a unit sphere in various dimensions. The species of polytopes are regulated by the network architecture, such as the number of units and layers. For a variety of activation functions, generalization of polytopes is obtained, which we call neural polytopes. They are a smooth analogue of polytopes, exhibiting geometric duality. This finding initiates research of generative discrete geometry to approximate surfaces by machine learning.
Submission Number: 6