Exchangeable Models in Meta Reinforcement Learning

Iryna Korshunova, Jonas Degrave, Joni Dambre, Arthur Gretton, Ferenc Huszar

Student First Author: Yes

Keywords: meta-RL, task inference, exchangeability

Abstract: One recent approach to meta reinforcement learning (meta-RL) is to integrate models for task inference with models for control. The former component is often based on recurrent neural networks, which do not directly exploit the exchangeable structure of the inputs. We propose to use a lightweight, yet an expressive architecture that accounts for exchangeability. Combined with an off-policy reinforcement learning algorithm, it results in a meta-RL method that is sample-efficient, fast to train and able to quickly adapt to new test tasks as demonstrated on a couple of widely used benchmarks.

TL;DR: a meta-RL method that combines an exchangeable model for task inference with an off-policy RL algorithm for control