Cross-Modal Domain Adaptation for Reinforcement Learning

Xiong-Hui Chen, Shengyi Jiang, Feng Xu, Yang Yu

Keywords: Domain Adaptation, Reinforcement Learning

Abstract: Domain adaptation is a promising direction for deploying RL agents in real-world applications, where vision-based robotics tasks constitute an important part. Cur-rent methods that train polices on simulated images not only require a delicately crafted simulator, but also add extra burdens to the training process. In this paper, we propose a method that can learn a mapping from high-dimensional images to low-level simulator states, allowing agents trained on the source domain of state input to transfer well to the target domain of image input. By fully leveraging the sequential information in the trajectories and incorporating the policy to guide the training process, our method overcomes the intrinsic ill-posedness in cross-modal domain adaptation when structural constraints from the same modality are unavailable. Experiments on MuJoCo environments show that the policy, once combined with the mapping function, can be deployed directly in the target domain with only a small performance gap, while current methods designed for same-modal domain adaptation fail on this problem.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

One-sentence Summary: By fully leveraging the sequential information in the trajectories and incorporating the policy to guide the training process, our method realizes cross-modal domain adaptation in RL settings.

Reviewed Version (pdf): https://openreview.net/references/pdf?id=3EVSBnsVYe