Sparse, Geometric Autoencoder Models of V1

Jonathan Raymond Huml, Abiy Tasissa, Demba E. Ba

Keywords: Locality, Manifold Learning, Graph Laplacian, Phase Symmetry

TL;DR: Recurrent sparse autoencoders learn brain-like Gabor filters when adding additional regularization that captures physical constraints of V1

Abstract: The classical sparse coding model represents visual stimuli as a convex combination of a handful of learned basis functions that are Gabor-like when trained on natural image data. However, the Gabor-like filters learned by classical sparse coding far overpredict well-tuned simple cell receptive field (SCRF) profiles. A number of subsequent models have either discarded the sparse dictionary learning framework entirely or have yet to take advantage of the surge in unrolled, neural dictionary learning architectures. A key missing theme of these updates is a stronger notion of \emph{structured sparsity}. We propose an autoencoder architecture whose latent representations are implicitly, locally organized for spectral clustering, which begets artificial neurons better matched to observed primate data. The weighted-$\ell_1$ (WL) constraint in the autoencoder objective function maintains core ideas of the sparse coding framework, yet also offers a promising path to describe the differentiation of receptive fields in terms of a discriminative hierarchy in future work.