Sparse Deep Scattering Croisé Network

Romain Cosentino, Randall Balestriero, Richard Baraniuk, Ankit Patel

Abstract:

In this work, we propose the Sparse Deep Scattering Croisé Network (SDCSN) a novel architecture based on the Deep Scattering Network (DSN). The DSN is achieved by cascading wavelet transform convolutions with a complex modulus and a time-invariant operator. We extend this work by first,
crossing multiple wavelet family transforms to increase the feature diversity while avoiding any learning. Thus providing a more informative latent representation and benefit from the development of highly specialized wavelet filters over the last decades. Beside, by combining all the different wavelet representations, we reduce the amount of prior information needed regarding the signals at hand.
Secondly, we develop an optimal thresholding strategy for over-complete filter banks that regularizes the network and controls instabilities such as inherent non-stationary noise in the signal. Our systematic and principled solution sparsifies the latent representation of the network by acting as a local mask distinguishing between activity and noise. Thus, we propose to enhance the DSN by increasing the variance of the scattering coefficients representation as well as improve its robustness with respect to non-stationary noise.
We show that our new approach is more robust and outperforms the DSN on a bird detection task.

TL;DR: We propose to enhance the Deep Scattering Network in order to improve control and stability of any given machine learning pipeline by proposing a continuous wavelet thresholding scheme

Keywords: Deep Scattering Network, Continuous Wavelet Thresholding, Sparse Activations, Time-frequency represenation, Multi-Family, Wavelets, Convolutional Network, Bird Detection