PCA-Enhanced Autoencoders for Nonlinear Dimensionality Reduction in Low Data Regimes

Muhammad Al-Digeil, Yuri Grinberg, Daniele Melati, Jens H. Schmid, Pavel Cheben, Siegfried Janz, Dan-Xia Xu

Abstract: Many scientific domains, such as nanophotonic design, gene expression, and materials design, are limited by high costs of acquiring data. This data is often intrinsically low-dimensional, nonlinear, and benefits from dimensionality reduction. Autoencoders (AE) provide nonlinear dimensionality reduction but are typically ineffective for low data regimes. Principal Component Analysis (PCA) is data-efficient but limited to linear dimensionality reduction. We propose a technique that harnesses the benefits of both methods by using PCA to initialize an AE. The proposed approach outperforms both PCA and standard AEs in low-data regimes and is comparable to the best of either of the two in other scenarios.