World Models as Reference Trajectories for Rapid Motor Adaptation

Carlos Stein Brito, Daniel C McNamee

Keywords: Motor Adaptation, Model-based RL, Rapid Adaptation, Model Reference Control, Data Efficiency, Naturalistic Motor Control

TL;DR: Dual control framework that uses world model predictions as implicit reference trajectories for rapid adaptation

Abstract: Learned control policies often fail when deployed in real-world environments with changing dynamics. When system dynamics shift unexpectedly, performance degrades until models are retrained on new data. We introduce Reflexive World Models (RWM), a dual control framework that uses world model predictions as implicit reference trajectories for rapid adaptation. Our method separates the control problem into long-term reward maximization through reinforcement learning and robust motor execution through reward-free rapid control in latent space. This dual architecture achieves significantly faster adaptation with low online computational cost compared to model-based RL baselines, while maintaining near-optimal performance. The approach combines the benefits of flexible policy learning through reinforcement learning with rapid error correction capabilities, providing a theoretically grounded method for maintaining performance in high-dimensional continuous control tasks under varying dynamics.

Primary Area: Reinforcement learning (e.g., decision and control, planning, hierarchical RL, robotics)

Submission Number: 13023