Compressed information is all you need: unifying intrinsic motivations and representation learningDownload PDF

Published: 21 Nov 2022, Last Modified: 05 May 2023InfoCog @ NeurIPS 2022 PosterReaders: Everyone
Keywords: guiding principle, information theory, representation learning, intrinsic motivation
TL;DR: We propose a quantified information theoretic guiding principle that unifies works from representation learning and intrinsic motivations.
Abstract: Humans can recognize categories, shapes, colors, grasp/manipulate objects, run or take a plane. To reach this level of cognition, developmental psychology identifies two key elements: 1- children have a spontaneous drive to explore and learn open-ended skills, called intrinsic motivation; 2- perceiving and acting are deeply intertwined: a chair is a chair because I can sit on it. This supports the hypothesis that the development of perception and skills may be continually underpinned by one guiding principle. Here, we investigate the consequence of maximizing the multi-information of a simple cognitive architecture, modelled as a causal model. We show that it provides a coherent unifying view on numerous results in unsupervised learning of representations and intrinsic motivations. This poses our framework as a serious candidate to be a guiding unifying principle.
0 Replies