Zero-Shot Adaptation of Behavioral Foundation Models to Unseen Dynamics

Maksim Bobrin, Ilya Zisman, Alexander Nikulin, Dmitry V. Dylov, Vladislav Kurenkov

Keywords: zero-shot RL, successor features, successor measure, forward-backward representations

TL;DR: We provide both theoretical and empirical evidence that Forward–Backward representations cannot adapt to changing dynamics and introduce a method that overcomes this, generalizing to both seen and unseen dynamics.

Abstract: Behavioral Foundation Models (BFMs) like successor measure-based methods excel in zero-shot policy generation but struggle with dynamic changes, limiting real-world applicability (\eg, robotics). We show that Forward-Backward (FB) representations fail to distinguish between different dynamics, causing latent interference. To fix this, we propose an FB model with a transformer-based belief estimator, enabling better zero-shot adaptation. Additionally, clustering the policy space by dynamics improves performance. Our method adapts to trained dynamics and generalizes to unseen ones, achieving up to 2x higher zero-shot returns in discrete and continuous tasks compared to baselines.

Submission Number: 68