Autoencoder with Distribution Preservation

Hang Zhang, Wei-Jie Xu, Kai Ming Ting

Keywords: Auto-encoder, Dimensionality reduction, Distribution preservation

Abstract: This paper proposes an improved autoencoder method. On the basis of maintaining the reconstruction accuracy, we introduce a data distribution preservation mechanism to improve the performance of the dimensionality reduction of the model. Traditional autoencoders only focus on the point-to-point distance between the input sample and its reconstruction result, ignoring the preservation of the overall distribution structure of the data. To solve this problem, we introduced the Kernel Mean Embedding (KME) term based on a kernel function with good topological properties into the loss function to measure the difference between the original data distribution and the reconstructed data distribution. This method effectively maintains the topological features and distribution characteristics of the global data. Experimental results show that compared with traditional autoencoders and existing topological autoencoders, our method performs better on multiple datasets, especially in terms of dimensionality reduction quality and structural preservation of latent representations.

Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Submission Number: 7648