Path Divergence Objective: Boundedly-Rational Decision Making in Partially Observable Environments

Tomáš Gavenčiak, David Hyland, Lancelot Da Costa, Michael J. Wooldridge, Jan Kulveit

Keywords: bounded rationality, rational inattention, agency, POMDP, active inference

TL;DR: We introduce PDO, a novel model of boundedly-rational decision-making in partially-observable environments, and analyze its properties such as information-seeking, and its connections to active inference and traditional decision making.

Abstract: We introduce the Path Divergence Objective (PDO), a novel model of boundedly-rational decision-making in stochastic, partially-observable environments. The PDO is derived from fundamental physical principles, including embodiment and the inherent costs of information processing. This framework enables us to model key features observed in real-world agent behavior, such as curiosity-driven exploration, novelty-seeking, and the intention-behavior gap. By adjusting a single parameter, the PDO can describe a continuous spectrum of decision-making strategies, ranging from highly irrational to perfectly rational. This flexibility makes the PDO applicable to a wide range of scenarios, including modeling biological organisms, simulating interactions between agents with varying degrees of bounded rationality, addressing AI alignment challenges, and designing AI systems that interact more effectively with humans.

Submission Number: 101