A Recursive Algorithm for Computing Inferences in Imprecise Markov Chains

Thomas Krak, Natan T'Joens, Jasper De Bock, Gert De Cooman

Abstract:

We present an algorithm that can efficiently compute a broad
class of inferences for discrete-time imprecise Markov chains, a gener-
alised type of Markov chains that allows one to take into account par-
tially specified probabilities and other types of model uncertainty. The
class of inferences that we consider contains, as special cases, tight lower
and upper bounds on expected hitting times, on hitting probabilities and
on expectations of functions that are a sum or product of simpler ones.
Our algorithm exploits the specific structure that is inherent in all these
inferences: they admit a general recursive decomposition. This allows us
to achieve a computational complexity that scales linearly in the number
of time points on which the inference depends, instead of the exponential
scaling that is typical for a naive approach.