Measuring Goal-Directedness

Matt MacDermott, James Fox, Francesco Belardinelli, Tom Everitt

Keywords: Causality, Graphical Models, Maximum Causal Entropy, Agency

TL;DR: We propose a formal measure of goal-directedness in probabalistic graphical models, by adapting and generalising the maximum causal entropy framework.

Abstract: We define _maximum entropy goal-directedness (MEG)_, a formal measure of goal-directedness in causal models and Markov decision processes, and give algorithms for computing it. Measuring goal-directedness is important, as its a critical element of many concerns about harm from AI. It is also of philosophical interest, as goal-directedness is a key aspect of agency. MEG is based on an adaption of the maximum causal entropy framework used in inverse reinforcement learning. It can be used to measures goal-directedness with respect to a known utility function, a hypothesis class of utility functions, or a set of random variables. We prove that MEG satisfies several desiderata, and demonstrate our algorithms in preliminary experiments.

Submission Number: 98