Self-Organization adds application robustness to deep learners

Pitoyo Hartono, Thomas Trappenberg

Abstract: While self-organizing principles have motivated much of early learning models, such principles have rarely been included in deep learning architectures. Indeed, from a supervised learning perspective it seems that topographic constraints are rather decremental to optimal performance. Here we study a network model that incorporates self-organizing maps into a supervised network and show how gradient learning results in a form of a self-organizing learning rule. Moreover, we show that such a model is robust in the sense of its application to a variety of areas, which is believed to be a hallmark of biological learning systems.

TL;DR: integration of self-organization and supervised learning in a hierarchical neural network

Keywords: supervised learning, unsupervised learning, self-organization, internal representation, topological structure