Gradient-based Training of Slow Feature Analysis by Differentiable Approximate Whitening

Merlin Schüler, Hlynur Davíð Hlynsson, Laurenz Wiskott

Sep 27, 2018 ICLR 2019 Conference Blind Submission readers: everyone Show Bibtex
  • Abstract: We propose Power Slow Feature Analysis, a gradient-based method to extract temporally slow features from a high-dimensional input stream that varies on a faster time-scale, as a variant of Slow Feature Analysis (SFA). While displaying performance comparable to hierarchical extensions to the SFA algorithm, such as Hierarchical Slow Feature Analysis, for a small number of output-features, our algorithm allows fully differentiable end-to-end training of arbitrary differentiable approximators (e.g., deep neural networks). We provide experimental evidence that PowerSFA is able to extract meaningful and informative low-dimensional features in the case of (a) synthetic low-dimensional data, (b) visual data, and also for (c) a general dataset for which symmetric non-temporal relations between points can be defined.
  • Keywords: Slow Feature Analysis, Deep Learning, Spectral Embedding, Temporal Coherence
  • TL;DR: We propose a way to train Slow Feature Analysis with stochastic gradient descent eliminating the need for greedy layer-wise training.
0 Replies