Operator Bellman Equations Support Compositional & Verifiable Planning in Hierarchical World-models

Tom Ringstrom

Confirmation: Yes

Keywords: N/A

Abstract: We introduce new reward-free Bellman Equations called Operator Bellman Equations which, rather than value functions, produce predictive planning representations called state-time feasibility functions (STFFs) which are compositional, factorizable, and interpretable. This means: 1) STFFs can be sequentially composed to compute high-dimensional predictions over long-horizons of sequential Options (policies), 2) high-dimensional STFFs can be represented and computed efficiently in a factorized form, and 3) STFFs record the probabilities of semantically interpretable goal-success and constraint-violation events. We discuss how these properties are critical for verifiable planning that can scale to dynamic high-dimensional world-models.

Submission Number: 18