Recursive Reasoning with Neural NetworksDownload PDF

01 Mar 2023 (modified: 30 May 2023)Submitted to Tiny Papers @ ICLR 2023Readers: Everyone
Keywords: neural algorithmic reasoning, graph neural networks
TL;DR: Neural networks cannot reason recursively so we propose an architecture that enables them to do this and show that our method benefits generalization on depth-first search.
Abstract: Many problems can naturally be thought about recursively. However, neural networks fundamentally cannot reason this way on arbitrarily large problems. This is because they do not have the memory to maintain state for the maximum recursion depth required. Solving this issue would enable neural networks to reason like a wide range of classical recursive algorithms (e.g., tree search in model-based RL). To address this, we propose a neural architecture augmented with a stack that learns to save and recall state as needed. We empirically demonstrate the utility of this method on a recursive neural algorithmic reasoning task (learning depth-first search) and show that our architecture leads to improved generalization.
9 Replies

Loading