Discriminative reconstruction via simultaneous dense and sparse coding

Abiy Tasissa, Manos Theodosis, Bahareh Tolooshams, Demba E. Ba

Abstract:

Discriminative features extracted from the sparse coding model have been shown to perform well for classification. Recent deep learning architectures have further improved reconstruction in inverse problems by considering new dense priors learned from data. We propose a
novel dense and sparse coding model that integrates both representation capability and discriminative features. The model studies the problem of recovering a dense vector x and a sparse vector u given measurements of the form y = Ax+Bu. Our first analysis relies
on a geometric condition, specifically the minimal angle between the spanning subspaces of matrices A and B, which ensures a unique solution to the model. The second analysis shows that, under some conditions on A and B, a convex program recovers the dense and sparse components. We validate the effectiveness of the model on simulated data and propose a dense and sparse autoencoder (DenSaE) tailored to learning the dictionaries from the dense and sparse model. We demonstrate that (i) DenSaE denoises natural images better than architectures derived from the sparse coding model (Bu), (ii) in the presence of noise, training the biases in the latter amounts to implicitly learning the Ax + Bu model, (iii) A and B capture low- and high-frequency contents, respectively, and (iv) compared to the sparse coding model, DenSaE offers a balance between discriminative power and representation.

Submission Length: Long submission (more than 12 pages of main content)

Previous TMLR Submission Url: https://openreview.net/forum?id=MmfBxdMDoo

Changes Since Last Submission: We appreciate the feedback from the reviewers. In this final version, we have revised the paper to enhance its presentation and clarity. We have also included the links to the code that reproduce the numerical experiments in the paper.

Code: https://github.com/manosth/densae/ and https://github.com/btolooshams/densae

Assigned Action Editor: ~Moshe_Eliasof1

Submission Number: 2406