A Theoretical Analysis of Information Bottlenecks for Zero-Shot Transfer in Reinforcement Learning

Kenzo Clauw, Daniel Polani, Nicola Catenacci Volpi

Track: Research Track

Keywords: Information Bottleneck, State Abstraction, Transfer Learning, Information Theory

Abstract: This work studies zero-shot policy transfer in reinforcement learning (RL), where a policy trained on multiple tasks is deployed on new tasks without fine-tuning. We study the Information Bottleneck (IB) framework as a probabilistic state abstraction, compressing observations into latent variables that retain task-relevant information. Our goal is to theoretically understand when and why such compressed representations enable policy transfer. We make three contributions: (1) an encoder-free metric that measures a priori task transferability using the Shannon divergence between action distributions, jointly with bisimulation distances comparing dynamics and rewards, (2) an encoder-dependent diagnostic metric, Latent-Action Divergence (LAD), which compares action distributions in the latent space to assess how well the learned abstraction preserves transferable behavior, and (3) a generalization bound showing that excess information, information retained in the latent representation that is not predictive of the policy, bounds the generalization error. Finally, we discuss empirically testable ideas motivated by our theoretical framework.

Submission Number: 98