Deep Network Classification by Scattering and Homotopy Dictionary Learning

John Zarka, Louis Thiry, Tomas Angles, Stephane Mallat

Code: [ j-zarka/SparseScatNet](https://github.com/j-zarka/SparseScatNet)

Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/arxiv:1910.03561/code)

TL;DR: A scattering transform followed by supervised dictionary learning reaches a higher accuracy than AlexNet on ImageNet.

Abstract:

We introduce a sparse scattering deep convolutional neural network, which provides a simple model to analyze properties of deep representation learning for classification. Learning a single dictionary matrix with a classifier yields a higher classification accuracy than AlexNet over the ImageNet 2012 dataset. The network first applies a scattering transform that linearizes variabilities due to geometric transformations such as translations and small deformations.
A sparse $\ell^1$ dictionary coding reduces intra-class variability while preserving class separation through projections over unions of linear spaces. It is implemented in a deep convolutional network with a homotopy algorithm having an exponential convergence. A convergence proof is given in a general framework that includes ALISTA. Classification results are analyzed on ImageNet.

Keywords: dictionary learning, scattering transform, sparse coding, imagenet