Nonlinear Extensions of Reconstruction ICA

Apaar Sadhwani

Abstract: In a recent paper [1] it was observed that unsupervised feature learning with overcomplete features could be achieved using linear autoencoders (named Reconstruction Independent Component Analysis). This algorithm has been shown to outperform other well-known algorithms by penalizing the lack of diversity (or orthogonality) amongst features. In our project, we wish to extend and improve this algorithm to include other non-linearities. In this project we have considered three unsupervised learning algorithms:(a) Sparse Autoencoder (b) Reconstruction ICA (RICA), a linear autoencoder proposed in [1], and (c) Nonlinear RICA, a proposed extension of RICA for capturing nonlinearities in feature detection.