${lil}$Gym: Natural Language Visual Reasoning with Reinforcement LearningDownload PDF

Published: 21 Oct 2022, Last Modified: 05 May 2023LaReL 2022Readers: Everyone
Keywords: reinforcement learning, natural language, visual reasoning, benchmark
TL;DR: A new benchmark for language-conditioned reinforcement learning in visual environments with highly compositional human-written language.
Abstract: We present ${lil}$Gym, a new benchmark for language-conditioned reinforcement learning in visual environments. ${lil}$Gym is based on 2,661 highly-compositional human-written natural language statements grounded in an interactive visual environment. We annotate all statements with executable Python programs representing their meaning to enable exact reward computation in every possible world state. Each statement is paired with multiple start states and reward functions to form thousands of distinct Markov Decision Processes of varying difficulty. We experiment with ${lil}$Gym with different models and learning regimes. Our results and analysis show that while existing methods are able to achieve non-trivial performance, ${lil}$Gym forms a challenging open problem. ${lil}$Gym is available at https://lil.nlp.cornell.edu/lilgym/.
3 Replies

Loading