Dynamics Are Learned, Not Told: Semi-Supervised Discovery of Latent Dynamics Geometries For Zero-Shot Policy Adaptation

Zhiming Xu, Weitao Zhou, Xianghui Pan, Nanshan Deng, Chengju Liu, Qijun Chen, Chenpeng Yao

TL;DR: By controlling the Lipschitz smoothness and latent topology of a trajectory dynamics encoder via contrastive learning, this work provides both theoretical guarantees and practical zero-shot robustness under dynamics shifts.

Abstract: Real-world dynamics shifts pose a critical challenge for reinforcement learning, yet prior methods typically rely on encoding explicitly identified physical parameters into a latent context, a rigid parameterization that proves brittle to unmodeled or compound dynamics variations. We instead investigate dynamics adaptation through the lens of latent geometry, and show theoretically that target-domain regret is controlled by the Lipschitz smoothness of a trajectory dynamics encoder. We further prove that this Lipschitz constant can be upper-bounded through optimizing a multi-positive InfoNCE objective, yielding a smooth, task-relevant latent topology without privileged dynamics information. On MuJoCo benchmarks, our method significantly outperforms explicit identification baselines under severe dynamics shifts, including unmodeled structural failures, while simultaneously improving in-distribution stability and latent interpretability. Overall, these results validate that controlling latent smoothness is a principled and scalable mechanism for robust adaptation.

Lay Summary: Robots trained in simulation or controlled environments often struggle when real-world conditions change, such as when a robot becomes heavier or a motor weakens. Many existing methods try to handle this by explicitly estimating physical properties of the system, but this approach can break down when changes are complex or not well described. In this work, instead of explicitly identifying what has changed, we train the robot to directly learn how changes in its environment affect the outcomes of its actions. This leads to a more flexible way of adapting to new conditions. We develop a learning method that encourages the robot to organize its experiences in a smooth and structured way, which we show is important for reliable adaptation. In experiments on standard robotics simulation tasks, our method consistently performs better than existing approaches, especially when conditions change in unexpected or difficult ways. It also produces more stable behavior and more interpretable internal representations. Overall, our results suggest that learning structured and well-organized internal representations is a key ingredient for building robots that can adapt reliably to changing real-world conditions.

Link To Code: https://github.com/Mr-Wonderfool/Latent-Dynamics-Geometries

Primary Area: Applications->Robotics

Keywords: Reinforcement Learning, Dynamics Adaptation, Contrastive Learning, Latent Geometry

Submission Number: 34424