The Union of Manifolds HypothesisDownload PDF

Published: 07 Nov 2022, Last Modified: 05 May 2023NeurReps 2022 PosterReaders: Everyone
Keywords: manifold hypothesis
TL;DR: We argue that the manifold hypothesis is incomplete, and that data often lies on a union of manifolds, which we empirically verify
Abstract: The manifold hypothesis states that low-dimensional manifold structure exists in high-dimensional data, which is strongly supported by the success of deep learning in processing such data. However, we argue here that the manifold hypothesis is incomplete, as it does not allow any variation in the intrinsic dimensionality of different sub-regions of the data space. We thus posit the union of manifold hypothesis, which states that high-dimensional data of interest comes from a union of disjoint manifolds; this allows intrinsic dimensionality to vary. We empirically verify this hypothesis on image datasets using a standard estimator of intrinsic dimensionality, and also demonstrate an improvement in classification performance derived from this hypothesis. We hope our work will encourage the community to further explore the benefits of considering the union of manifolds structure in data.
4 Replies