mTSBench: Benchmarking Multivariate Time Series Anomaly Detection and Model Selection at Scale

TMLR Paper6187 Authors

Abstract: Anomaly detection in multivariate time series is essential across domains such as healthcare, cybersecurity, and industrial monitoring, yet remains fundamentally challenging due to high-dimensional dependencies, the presence of cross-correlations between time-dependent variables, and the scarcity of labeled anomalies. We introduce mTSBench, the largest benchmark to date for multivariate time series anomaly detection and model selection, consisting of 344 labeled time series across 19 datasets from a wide range of application domains. We comprehensively evaluate 24 anomaly detectors, including the only two publicly available large language model-based methods for multivariate time series. Consistent with prior findings, we observe that no single detector dominates across datasets, motivating the need for effective model selection. We benchmark three recent model selection methods and find that even the strongest of them remain far from optimal. Our results highlight the outstanding need for robust, generalizable selection strategies. We open-source the benchmark at \url{https://tinyurl.com/mTSBench} to encourage future research.

Submission Length: Regular submission (no more than 12 pages of main content)

Changes Since Last Submission: edited description for dataset MITDB and SVDB, colored in blue

Assigned Action Editor: ~Min_Wu2

Submission Number: 6187