Successive Refinement and Coarsening of the Information BottleneckDownload PDF

Published: 21 Nov 2022, Last Modified: 05 May 2023InfoCog @ NeurIPS 2022 PosterReaders: Everyone
Keywords: Incremental learning, Multi-Stage information processing, Information Bottleneck, Successive Refinement
Abstract: We study two central aspects of information processing in cognitive systems: one is the ability to incorporate fresh information to already learnt models; the other is the ``trickling'' of information through the many layers of a cognitive processing pipeline. We investigate the extent to which these specific structures of cognitive processing impact their informational optimal limits. To do so, we present mathematical characterisations and low-dimensional numerical examples, which explore formal properties of the Information Bottleneck method: namely, how it relates to successive refinement, and successive coarsening of information.
In-person Presentation: yes
0 Replies