Context Transfer and Q-Transferable Tasks

Amin Mousavi, Babak Nadjar Araabi, Majid Nili Ahmadabadi

Abstract: This article discusses the notion of context transfer in reinforcement learning tasks. Context transfer, as defined in this article, implies knowledge transfer between tasks that share the same environment's dynamics and reward function, but have different state and action spaces. For example, we have a working mobile robot in an environment. At some point, we decide to upgrade its sensors and/or actuators. Any change in these modules will result in a different description of the agent-environment model, and the trained knowledge is no longer applicable. We consider the tasks of the old and new robots, as the source and target tasks, respectively. The Markov decision process (MDP) of these tasks, under certain conditions, are called Q-transferable tasks, and the problem of knowledge transfer between them is called context transfer. We investigate the relation of the MDPs of these tasks.