Zero-Shot Policy Transfer with Disentangled Attention

Josh Roy, George Konidaris

Keywords: Transfer Learning, Reinforcement Learning, Attention, Domain Adaptation, Representation Learning, Feature Extraction

TL;DR: We present an agent that uses a beta-vae to extract visual features and an attention mechanism to ignore irrelevant features from visual observations to enable robust transfer between visual domains.

Abstract: Domain adaptation is an open problem in deep reinforcement learning (RL). Often, agents are asked to perform in environments where data is difficult to obtain. In such settings, agents are trained in similar environments, such as simulators, and are then transferred to the original environment. The gap between visual observations of the source and target environments often causes the agent to fail in the target environment. We present a new RL agent, SADALA (Soft Attention DisentAngled representation Learning Agent). SADALA first learns a compressed state representation. It then jointly learns to ignore distracting features and solve the task presented. SADALA's separation of important and unimportant visual features leads to robust domain transfer. SADALA outperforms both prior disentangled-representation based RL and domain randomization approaches across RL environments (Visual Cartpole and DeepMind Lab).

Code: https://drive.google.com/open?id=1KRaAjLofmAGpk2OhsmjUaXG1-DExkpOy