Multi-Cost-Bounded Reachability Analysis of POMDPs

Alexander Bork, Joost-Pieter Katoen, Tim Quatmann, Svenja Stein

Keywords: POMDP, Partial Observability, Markov Decision Process, Cost-Bounded Reachability, Probabilistic Model Checking, Formal Methods

TL;DR: We present a general framework to approximate the maximal probability to reach a target while satisfying constraints on the accumulated costs for multiple cost measures in POMDPs.

Abstract:

We consider multi-dimensional cost-bounded reachability probability objectives for partially observable Markov decision processes (POMDPs). The goal is to compute the maximal probability to reach a set of target states while simultaneously satisfying specified bounds on incurred costs.
Such objectives generalise well-studied POMDP objectives by allowing multiple upper and lower bounds on different cost or reward measures, e.g. to naturally model scenarios where an agent acts under limited resources.

We present a reduction of the multi-cost-bounded problem to unbounded reachability probabilities on an unfolding of the original POMDP.
We employ a refined approach in case the agent is cost-aware—i.e., collected costs are fully observed—and also consider a setting where only partial information about the collected costs is known.
Our approaches elegantly lift existing results from the fully observable MDP case to POMDPs.
An empirical evaluation shows the potential of analysing POMDPs under multi-cost-bounded reachability objectives in practical settings.

Code Link: https://zenodo.org/records/15642233

Readers: auai.org/UAI/2025/Conference, auai.org/UAI/2025/Conference/Area_Chairs, auai.org/UAI/2025/Conference/Reviewers, auai.org/UAI/2025/Conference/Submission523/Authors, auai.org/UAI/2025/Conference/Submission523/Reproducibility_Reviewers

Submission Number: 523