Forward Prediction for Physical Reasoning

Rohit Girdhar, Laura Gustafson, Aaron B. Adcock, Laurens van der Maaten

Keywords: Forward prediction, physical reasoning

Abstract: Physical reasoning requires forward prediction: the ability to forecast what will happen next given some initial world state. We study the performance of state-of-the-art forward-prediction models in the complex physical-reasoning tasks of the PHYRE benchmark (Bakhtin et al., 2019). We do so by incorporating models that operate on object or pixel-based representations of the world into simple physical-reasoning agents. We find that forward-prediction models can improve physical-reasoning performance, particularly on complex tasks that involve many objects. However, we also find that these improvements are contingent on the test tasks being small variations of train tasks, and that generalization to completely new task templates is challenging. Surprisingly, we observe that forward predictors with better pixel accuracy do not necessarily lead to better physical-reasoning performance. Nevertheless, our best models set a new state-of-the-art on the PHYRE benchmark.

One-sentence Summary: We experiment with forward prediction models through the lens of a challenging new physical reasoning benchmark, PHYRE. We establish a new state of the art, and report rigorous analysis on where these models work and where future work is needed.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/arxiv:2006.10734/code)

Reviewed Version (pdf): https://openreview.net/references/pdf?id=HmM9LixdT1