Approximating Labeled Markov Processes

Josée Desharnais, Vineet Gupta, Radha Jagadeesan, Prakash Panangaden

Abstract: We study approximate reasoning about continuous-state labeled Markov processes. We show how to approximate a labeled Markov process by a family of finite-state labeled Markov chains. We show that the collection of labeled Markov processes carries a Polish space structure with a countable basis given by finite state Markov chains with rational probabilities. The primary technical tools that we develop to reach these results are: a finite-model theorem for the modal logic used to characterize bisimulation; and a categorical equivalence between the category of Markov processes (with simulation morphisms) with the /spl omega/-continuous dcpo Proc, defined as the solution of the recursive domain equation Proc=/spl Pi//sub Labels/ P/sub Prob/(Proc). The correspondence between labeled Markov processes and Proc yields a logic complete for reasoning about simulation for continuous-state processes.