Decision Making as Language Generation

Roland Memisevic, Sunny Panchal, Mingu Lee

Keywords: foundation models, decision transformers, transfer learning

TL;DR: We study the challenges and benefits associated with treating decision making as pure language generation

Abstract: Decision transformers are a recently proposed approach to offline reinforcement learning that leverages transformer-based auto-regressive sequence models. We discuss challenges associated with fine-tuning a given, pre-trained language model on a decision making task. We propose solutions to these challenges and study their viability on a shortest path problem. We also show how given language model allows us to bring to bear data-centric approaches to improving the model and how it opens up the possibility to treat the decision transformer objective as one task alongside others to perform transfer learning.