Focus-Then-Reuse: Fast Adaptation in Visual Perturbation Environments

Jiahui Wang, Chao Chen, Jiacheng Xu, Zongzhang Zhang, Yang Yu

Keywords: reinforcement learning; visual domain adaptation

TL;DR: We introduce a novel object selection mechanism to allow sim-trained policies to rapidly adapt to real-world visual perturbations.

Abstract: Visual reinforcement learning has shown promise in various real-world applications. However, deploying policies in complex real-world environments with visual perturbations remains a significant challenge. We notice that humans tend to filter information at the object level prior to decision-making, facilitating efficient skill transfer across different contexts. Inspired by this, we introduce Focus-Then-Reuse (FTR), a method utilizing a novel object selection mechanism to focus on task-relevant objects, and directly reuse the simulation-trained policy on them. The training of the object selection mechanism integrates prior knowledge from a vision-language model and feedback from the environment. Experimental results on challenging tasks based on DeepMind Control Suite and Franka Emika Robotics demonstrate that FTR enables rapid adaptation in visual perturbation environments and achieves state-of-the-art performance. The source code is available at https://github.com/LAMDA-RL/FTR.

Primary Area: Reinforcement learning (e.g., decision and control, planning, hierarchical RL, robotics)

Submission Number: 19505