Optimization-Based Robust Permissive Synthesis for Interval MDPs

Khang Vo Huynh, David Parker, Lu Feng

Abstract:

We present an optimization-based framework for
robust permissive synthesis for Interval Markov Decision Processes (IMDPs), motivated by robotic decision-making under
transition uncertainty. In many robotic systems, model inaccuracies and sensing noise lead to interval-valued transition probabilities. While robust IMDP synthesis typically yields a single
policy and permissive synthesis assumes exact models, we show
that robust permissive synthesis under interval uncertainty can
be cast as a global mixed-integer linear program (MILP) that
directly encodes robust Bellman constraints. The formulation
maximizes a quantitative permissiveness metric (the number
of enabled state–action pairs), while guaranteeing that every
compliant strategy satisfies probabilistic reachability or expected reward specifications under all admissible transition
realizations. To address the exponential complexity of vertexbased uncertainty representations, we derive a dualizationbased encoding that eliminates explicit vertex enumeration and
scales linearly with the number of successors. Experimental
evaluation on four representative robotic benchmark domains
demonstrates scalability to IMDPs with hundreds of thousands
of states. The proposed framework provides a practical and
general foundation for uncertainty-aware, flexibility-preserving
controller synthesis in robotic systems.