Compressed information is all you need: unifying intrinsic motivations and representation learning

Arthur Aubret, Mathieu Lefort, Céline Teulière, Laetitia Matignon, Salima Hassas, Jochen Triesch

Keywords: guiding principle, information theory, representation learning, intrinsic motivation

TL;DR: We propose a quantified information theoretic guiding principle that unifies works from representation learning and intrinsic motivations.

Abstract: Humans can recognize categories, shapes, colors, grasp/manipulate objects, run or take a plane. To reach this level of cognition, developmental psychology identifies two key elements: 1- children have a spontaneous drive to explore and learn open-ended skills, called intrinsic motivation; 2- perceiving and acting are deeply intertwined: a chair is a chair because I can sit on it. This supports the hypothesis that the development of perception and skills may be continually underpinned by one guiding principle. Here, we investigate the consequence of maximizing the multi-information of a simple cognitive architecture, modelled as a causal model. We show that it provides a coherent unifying view on numerous results in unsupervised learning of representations and intrinsic motivations. This poses our framework as a serious candidate to be a guiding unifying principle.

In-person Presentation: no