Learning Multi-Timescale Phase-Amplitude Dynamics for Human-to-Robot Transfer: Toward Real-to-Sim-to-Real

Satoshi Yamamori, Jun Morimoto

Keywords: Imitation learning, Variational inference, Learning dynamical systems, Heterogeneous motion transfer

Abstract: Motion generation across heterogeneous embodiments is a significant challenge, and the Real-to-Sim-to-Real framework is a promising solution. Trajectory-based imitation learning requires learning dynamical models, i.e. simulation models, but previous approaches have struggled with representing both transient and steady dynamics across multiple timescales. We proposed a novel variational inference-based approach with phase-amplitude reduction to handle steady and transient behaviors in latent space. This framework can extend beyond imitation learning to heterogeneous embodiment transfer and enables future integration with reinforcement learning and foundation models for robust Real-to-Sim-to-Real framework.

Serve As Reviewer: ~Satoshi_Yamamori1

Submission Number: 21