Limit-sure Reachability for Small Memory Policies in POMDPs is NP-complete

Ali Asadi, Krishnendu Chatterjee, Raimundo Saona, Ali Shafiee

Keywords: Partially Observable Markov Decision Processes, Sequential Decision Making, Planning, Reachability Objectives, Computational Complexity

Abstract:

A standard model that arises in several applications in sequential decision-making is partially observable Markov decision processes (POMDPs) where a decision-making agent interacts with an uncertain environment. A basic objective in POMDPs is the reachability objective, where given a target set of states, the goal is to eventually arrive at one of them.

The limit-sure problem asks whether reachability can be ensured with probability arbitrarily close to 1. In general, the limit-sure reachability problem for POMDPs is undecidable. However, in many practical cases, the most relevant question is the existence of policies with a small amount of memory. In this work, we study the limit-sure reachability problem for POMDPs with a fixed amount of memory. We establish that the computational complexity of the problem is NP-complete.

Readers: auai.org/UAI/2025/Conference, auai.org/UAI/2025/Conference/Area_Chairs, auai.org/UAI/2025/Conference/Reviewers, auai.org/UAI/2025/Conference/Submission193/Authors, auai.org/UAI/2025/Conference/Submission193/Reproducibility_Reviewers

Submission Number: 193