Universal benchmark for actuation dynamics adaptation in reinforcement learning
Keywords: Transfer Learning, Domain Adaptation, Lifelong Learning, Continual Learning, Multi-Task Learning
TL;DR: A contribution to advance the understanding of reinforcement learning agents' transfer capabilities under actuation dynamic changes.
Abstract: Enabling reinforcement learning (RL) agents to adapt to changing environment dynamics is crucial for robustness. Consider the case where a robot's motors and gears change their behavior due to wear and tear over time, or where an old used-up part gets replaced.
Current literature primarily emphasizes resilience against observation noise, distractions in the environment or shifts in physical properties of the world. However, the problem of continual shifts or sudden changes in actuation dynamics is relatively unexplored. To facilitate systematic research in that regard, we contribute a Universal Benchmark for Actuation Dynamics Adaptation (UBADA). We present a universal set of wrappers compliant with the Gymnasium API standard, providing a multitude of challenges with continual (serial) and multi-task (parallel) learning scenarios of changing action dynamics.
We showcase the problem on visual and low-dimensional proprioceptive inputs, with dense or sparse rewards, utilizing the state-of-the-art learning algorithms Soft-Actor-Critic (SAC) and Data-regularized Q (DrQ).
Submission Number: 2
Loading