Informing Reinforcement Learning Agents by Grounding Natural Language to Markov Decision Processes

Benjamin Adin Spiegel, Ziyi Yang, William Jurayj, Katie Ta, Stefanie Tellex, George Konidaris

Keywords: reinforcement learning, natural language, language grounding, rl

TL;DR: We ground natural language to MDPs to improve the performance of RL agents.

Abstract: While significant efforts have been made to leverage natural language to accelerate reinforcement learning, utilizing diverse forms of language efficiently remains unsolved. Existing methods focus on mapping natural language to individual elements of MDPs such as reward functions or policies, but such approaches limit the scope of language they consider to make such mappings possible. We present an approach for leveraging general language advice by translating it to a grounded formal language capable of expressing information about *every* element of an MDP and its solution including policies, plans, reward functions, and transition functions. We also introduce a new model-based reinforcement learning algorithm called RLang-Dyna-Q that is capable of leveraging all such advice. We demonstrate the generality of our approach in a number of experiments by informing agents with a variety of natural language advice, leading to significant performance gains.

Primary Area: reinforcement learning

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 7423