Hierarchical Latent Dynamics Model with Multiple Timescales for Learning Long-Horizon Tasks

Kentaro Fujii, Shingo Murata

Abstract: Long-horizon tasks require intelligent agents, such as robots, to handle both temporal uncertainty and temporal dependency. Although world models have shown promise for solving tasks across many domains, they often struggle with managing long context in tasks due to the limited representation ability of their latent dynamics models. To overcome this issue, we propose a novel hierarchical latent dynamics model that takes into account multiple-timescale dynamics. Specifically, our proposed model, called the “multiple timescale recurrent state-space model” (MTRSSM), comprises a higher level with slow dynamics and a lower level with fast dynamics, each incorporating both deterministic and stochastic latent states. We demonstrate, both quantitatively and qualitatively, that a world model with our proposed MTRSSM can generate superior video predictions for long-horizon robotic object-manipulation tasks through latent imagination compared with other baselines. Importantly, we emphasize the critical role of the higher level in effectively handling temporal uncertainty and temporal dependency in long-horizon tasks. These findings indicate that the proposed MTRSSM enables intelligent agents to acquire a better understanding of the environment and generate more accurate predictions, thereby facilitating their learning and planning of long-horizon tasks.