Successive Refinement and Coarsening of the Information Bottleneck

Hippolyte Charvin, Daniel Polani, Nicola Catenacci Volpi

Keywords: Incremental learning, Multi-Stage information processing, Information Bottleneck, Successive Refinement

Abstract: We study two central aspects of information processing in cognitive systems: one is the ability to incorporate fresh information to already learnt models; the other is the ``trickling'' of information through the many layers of a cognitive processing pipeline. We investigate the extent to which these specific structures of cognitive processing impact their informational optimal limits. To do so, we present mathematical characterisations and low-dimensional numerical examples, which explore formal properties of the Information Bottleneck method: namely, how it relates to successive refinement, and successive coarsening of information.

In-person Presentation: yes