Minimum Curvature Manifold Learning

Yonghyeon Lee, Frank C. Park

Keywords: Autoencoder, Manifold, Curvature, Riemannian geometry

TL;DR: We propose a minimum extrinsic curvature principle for manifold regularization and Minimum Curvature Autoencoder (MCAE), a graph-free coordinate-invariant extrinsic curvature minimization framework for autoencoder regularization.

Abstract:

It is widely observed that vanilla autoencoders can have low manifold learning accuracy given a noisy or small training dataset.
Recent work has discovered that it is important to regularize the decoder that explicitly parameterizes the manifold,
where a neighborhood graph is employed for decoder regularization. However, one caveat of this method is that it is not always straightforward to construct a correct graph. Alternatively, one may consider naive graph-free regularization methods such as minimizing the norm of the decoder's Jacobian or Hessian, but these norms are not coordinate-invariant (i.e. reparametrization-invariant) and hence do not capture any meaningful geometric quantity of the manifold nor result in geometrically meaningful manifold regularization effects.
Another recent work called the isometric regularization implicitly forces the manifold to have zero intrinsic curvature, resulting in some geometrically meaningful regularization effects. But, since the intrinsic curvature does not capture how the manifold is embedded in the data space from an extrinsic perspective, the regularization effects are often limited. In this paper, we propose a {\it minimum extrinsic curvature principle} for manifold regularization and {\bf Minimum Curvature Autoencoder (MCAE)}, a graph-free coordinate-invariant extrinsic curvature minimization framework for autoencoder regularization. Experiments with various standard datasets demonstrate that MCAE improves manifold learning accuracy compared to existing methods, especially showing strong robustness to noise.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Submission Guidelines: Yes

Please Choose The Closest Area That Your Submission Falls Into: Generative models