Recursive Reasoning with Neural Networks

Jonas Jürß, Dulhan Hansaja Jayalath

Keywords: neural algorithmic reasoning, graph neural networks

TL;DR: Neural networks cannot reason recursively so we propose an architecture that enables them to do this and show that our method benefits generalization on depth-first search.

Abstract: Many problems can naturally be thought about recursively. However, neural networks fundamentally cannot reason this way on arbitrarily large problems. This is because they do not have the memory to maintain state for the maximum recursion depth required. Solving this issue would enable neural networks to reason like a wide range of classical recursive algorithms (e.g., tree search in model-based RL). To address this, we propose a neural architecture augmented with a stack that learns to save and recall state as needed. We empirically demonstrate the utility of this method on a recursive neural algorithmic reasoning task (learning depth-first search) and show that our architecture leads to improved generalization.