A Conservative Approach for Few-Shot Transfer in Off-Dynamics Reinforcement Learning

Paul Daoudi, CHRISTOPHE PRIEUR, Bogdan Robu, Merwan Barlier, Ludovic Dos Santos

Keywords: Reinforcement Learning, Simulator-based Reinforcement Learning, Off-Dynamics Reinforcement Learning

TL;DR: We propose a novel techniques for Few-Shot Transfer in Off-Dynamics Reinforcement Learning.

Abstract: Off-dynamics Reinforcement Learning (ODRL) seeks to transfer a policy from a source environment to a target environment characterized by distinct yet similar dynamics. In this context, traditional RL agents depend excessively on the dynamics of the source environment, resulting in the discovery of policies that excel in this environment but fail to provide reasonable performance in the target one. In the few-shot framework, a limited number of transitions from the target environment are introduced to facilitate a more effective transfer. Addressing this challenge, we propose a new approach inspired by recent advancements in Imitation Learning and conservative RL algorithms. The proposed method introduces a penalty to regulate the trajectories generated by the source-trained policy. We evaluate our method across various environments representing diverse off-dynamics conditions, where it demonstrates performance improvements compared to existing baselines across most tested scenarios.

Already Accepted Paper At Another Venue: already accepted somewhere else

Submission Number: 95