Gradient-based Training of Slow Feature Analysis by Differentiable Approximate Whitening

Merlin Schüler, Hlynur Davíð Hlynsson, Laurenz Wiskott

Abstract: We propose Power Slow Feature Analysis, a gradient-based method to extract temporally slow features from a high-dimensional input stream that varies on a faster time-scale, as a variant of Slow Feature Analysis (SFA). While displaying performance comparable to hierarchical extensions to the SFA algorithm, such as Hierarchical Slow Feature Analysis, for a small number of output-features, our algorithm allows fully differentiable end-to-end training of arbitrary differentiable approximators (e.g., deep neural networks). We provide experimental evidence that PowerSFA is able to extract meaningful and informative low-dimensional features in the case of (a) synthetic low-dimensional data, (b) visual data, and also for (c) a general dataset for which symmetric non-temporal relations between points can be defined.

Keywords: Slow Feature Analysis, Deep Learning, Spectral Embedding, Temporal Coherence

TL;DR: We propose a way to train Slow Feature Analysis with stochastic gradient descent eliminating the need for greedy layer-wise training.

Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/gradient-based-training-of-slow-feature/code)