Resolving Multiple-Dynamic Model Uncertainty in Hypothesis-Driven Belief-MDPs

Ofer Dagan, Tyler J. Becker, Zachary N. Sunberg

Abstract: This work explores the set of planning problems we define as hypothesis-driven planning. In these problems, a human with potentially no access to the system's underlying planning model (e.g., the reward or transition functions) is interested in answering a set of questions (hypotheses) that are not reflected in the underlying planning problem. To reason about the possible hypotheses and autonomously decide which one is most likely, we develop a new multiple-dynamic hypothesis belief Markov decision process (MDH-BMDP). This enables adding multiple hypotheses for existing planning problems and reasoning over arbitrary belief shapes using existing sparse tree search solvers. Lastly, we suggest a reward function that supports balancing the objectives of determining the correct hypothesis in time and performing well in the underlying problem; and test the framework's applicability and performance of the new reward function in simulations.

External IDs:dblp:conf/ifaamas/DaganBS25