Inferring the Future by Imagining the Past

Published: 21 Sept 2023, Last Modified: 02 Nov 2023NeurIPS 2023 spotlightEveryoneRevisionsBibTeX
Keywords: cognitive science, cogsci, inverse planning, Bayesian inference, theory of mind, Monte Carlo, inverse reinforcement learning
TL;DR: We model how humans infer an agent's goal from a snapshot of its current state. We frame the problem as Monte Carlo path tracing, which allows us to apply ideas from computer graphics to design a cognitively-plausible sample-efficient algorithm.
Abstract: A single panel of a comic book can say a lot: it can depict not only where the characters currently are, but also their motions, their motivations, their emotions, and what they might do next. More generally, humans routinely infer complex sequences of past and future events from a *static snapshot* of a *dynamic scene*, even in situations they have never seen before. In this paper, we model how humans make such rapid and flexible inferences. Building on a long line of work in cognitive science, we offer a Monte Carlo algorithm whose inferences correlate well with human intuitions in a wide variety of domains, while only using a small, cognitively-plausible number of samples. Our key technical insight is a surprising connection between our inference problem and Monte Carlo path tracing, which allows us to apply decades of ideas from the computer graphics community to this seemingly-unrelated theory of mind task.
Supplementary Material: zip
Submission Number: 12530
Loading