Text-based RL Agents with Commonsense Knowledge: New Challenges, Environments and Approaches

Keerthiram Murugesan, Mattia Atzeni, Pavan Kapanipathi, Pushkar Shukla, Sadhana Kumaravel, Gerald Tesauro, Kartik Talamadupula, Mrinmaya Sachan, Murray Campbell

Keywords: Reinforcement learning, Commonsense reasoning, Interactive fiction

TL;DR: A new interactive environment for building efficient RL agents for text-based games using commonsense knowledge.

Abstract: Text-based games have emerged as an important test-bed for Reinforcement Learning (RL) research, requiring RL agents to combine grounded language understanding with sequential decision making. In this paper, we examine the problem of infusing RL agents with commonsense knowledge. This allows agents to efficiently act in the world by pruning out implausible actions, and to perform look-ahead planning to determine how current actions might affect future world states. We design a new text-based gaming environment called TextWorld Commonsense (TWC) for training and evaluating RL agents with a specific kind of commonsense knowledge about objects, their attributes, and affordances. We introduce a number of RL agents that combine the sequential context with a dynamic graph representation of their beliefs of the world and commonsense knowledge from ConceptNet in different ways. We show that our agents act efficiently (fewer moves) and achieve better scores, and that learned policies can be transferred to other instances in TWC.