On forward recursive estimation for bivariate Markov chains

Yariv Ephraim, Brian L. Mark

Abstract: A bivariate Markov chain comprises a pair of finite-alphabet continuous-time random processes, which are jointly, but not necessarily individually, Markov. Forward recursive conditional mean estimators are developed for the state, the number of jumps from one state to another, and the total sojourn time of the process in each state. The recursions are implemented using Clark's transformation and tested in estimating the parameter of the bivariate Markov chain using the expectation-maximization (EM) algorithm.1