Change Point Detection via Variational Time-Varying Hidden Markov Model

Hao Zhang, Qishi Dong, Chuanlong Xie, Heng Peng

Primary Area: probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: Change Point Detection, Probabilistic Modeling, Bayesian Inference, Hidden Markov Model

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

Abstract: The task of modeling time series data that exhibit sudden regime shifts has been an enduring focus of research due to its inherent complexity. Among the various strategies to tackle this issue, the Hidden Markov Model (HMM) has been extensively investigated, which captures the regime changes by modeling the transition between latent states. Despite its popularity, the HMM-based methodology carries certain limitations, including specific distribution assumptions and its computational intensity for inference and learning, particularly when the number of change points is unidentified. In this work, we propose a novel approach that models the location of change points and introduce the $\textbf{TV-HMM}$, a variant of the Hidden Markov Model incorporating the time-varying location transition matrix. Based on the novel modeling scheme, we propose an associated variational EM algorithm that simultaneously detects the locations and the number of change points, together with inferring the posterior distributions of regime parameters. In contrast to previous approaches, the proposed method exhibits robustness against the misspecification of change point numbers and can be augmented with stochastic approximation techniques to effectively mitigate the computational burden. Furthermore, we establish the statistical consistency of the change point location estimation under the Gaussian likelihood assumption. We also generalize the parametric likelihood function using the Maximum Mean Discrepancy (MMD) and propose the semi-parametric $\textbf{TV-HMM}$ that is free of distribution assumptions. A series of experiments validate the theoretical convergence rate and demonstrate our estimation accuracy in terms of Rand index and MSE.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 3599