Learning Adaptive Diffusion Policies for Hybrid Dynamical Systems

Leroy D'Souza, Akash Karthikeyan, Yash Vardhan Pant, Sebastian Fischmeister

Keywords: Hybrid Systems, Reinforcement Learning, Mixture-of-Experts, Diffusion, Contrastive Learning

TL;DR: We propose a novel model-free, energy-diffusion based online RL algorithm that learns policies which can adapt to unseen scenarios in hybrid dynamical system control.

Abstract:

Hybrid dynamical systems result from the combination of a set of continuous-variable with a discrete-event system that can be influenced by the environment and encompasses systems like legged robots and aircrafts.
Model-based control of these systems is challenging, particularly when the systems are complex (high-dimensional non-linear dynamics) and the switching conditions are not perfectly known.
We propose a model-free actor-critic learning framework, MoE-Diff, that leverages energy-diffusion for multi-modal action generation with contrastive loss enabling scalable, compositional, and robust decision-making across complex tasks. We use MoE-Diff for control of hybrid systems in apriori unknown environments.
Qualitative and ablation studies attribute this improvement to MoE-Diff's ability to learn interpolatable representations and distinct behavior modes across experts enabling compositional generalization and adaptation to unseen scenarios.

Paper Type: New Full Paper

Submission Number: 4