OpenLock Grid: Learning Causal Structure with Deep RL Models

Yutong Yin, Xingbo Wang, Ziyin Xiong

Keywords: OpenLock, Causal reasoning, Reinforcement learning

Abstract: Causal reasoning is considered by many to be the cornerstone of human intelligence. Humans have the ability to develop causal schemas to solve novel problems and establish explanations of latent constraints while interacting with the environment. In this paper, we explore and examine agents' ability to discover causal schemas and generalize to more complex environments based on the OpenLock task, in which participants are required to escape a room by moving levers, and the sequential movements of the correct levers form a causal sequence beginning with either a common-cause (CC) or a common-effect (CE) structure. We develop a deep Q-learning model to solve the simplified OpenLock problem by learning the transition probability between solutions. We argue that the simplification is reasonable since the original task is a complex combination of several diverse sub-tasks and it's difficult to solve all sub-tasks in one single model. We focus on the agents' exploring the latent causal structure of levers, which is challenging and meaningful enough to show the potential of the Reinforcement Learning in causal reasoning. Experiments show that our method learns the underlying causal mechanisms perfectly and pertains a great generalization ability that can transfer the knowledge trained in simple conditions to more complex tasks.