$\ell$Gym: Natural Language Visual Reasoning with Reinforcement Learning

Anne Wu, Kianté Brantley, Noriyuki Kojima, Yoav Artzi

Keywords: reinforcement learning, natural language, visual reasoning, benchmark

TL;DR: A new benchmark for language-conditioned reinforcement learning in visual environments with highly compositional human-written language.

Abstract: We present $\ell$Gym, a new benchmark for language-conditioned reinforcement learning in visual environments. $\ell$Gym is based on 2,661 human-written natural language statements grounded in an interactive visual environment, and emphasizing compositionality and semantic diversity. We annotate all statements with Python programs representing their meaning. The programs are executable in an interactive visual environment to enable exact reward computation in every possible world state. Each statement is paired with multiple start states and reward functions to form thousands of distinct Markov Decision Processes of varying difficulty. We experiment with $\ell$Gym with different models and learning regimes. Our results and analysis show that while existing methods are able to achieve non-trivial performance, $\ell$Gym forms a challenging open problem.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Submission Guidelines: Yes

Please Choose The Closest Area That Your Submission Falls Into: Infrastructure (eg, datasets, competitions, implementations, libraries)