Hitting Times and Probabilities for Imprecise Markov Chains

Thomas Krak, Natan T'Joens, Jasper De Bock

Abstract:

We consider the problem of characterising expected
hitting times and hitting probabilities for imprecise
Markov chains. To this end, we consider three distinct
ways in which imprecise Markov chains have
been defined in the literature: as sets of homogeneous
Markov chains, as sets of more general stochastic processes,
and as game-theoretic probability models. Our
first contribution is that all these different types of imprecise
Markov chains have the same lower and upper
expected hitting times, and similarly the hitting probabilities
are the same for these three types. Moreover,
we provide a characterisation of these quantities that
directly generalises a similar characterisation for precise,
homogeneous Markov chains.