TimeSeAD: Benchmarking Deep Multivariate Time-Series Anomaly Detection

Dennis Wagner, Tobias Michels, Florian C.F. Schulz, Arjun Nair, Maja Rudolph, Marius Kloft

Abstract: Developing new methods for detecting anomalies in time series is of great practical significance, but progress is hindered by the difficulty of assessing the benefit of new methods, for the following reasons. (1) Public benchmarks are flawed (e.g., due to potentially erroneous anomaly labels), (2) there is no widely accepted standard evaluation metric, and (3) evaluation protocols are mostly inconsistent. In this work, we address all three issues: (1) We critically analyze several of the most widely-used multivariate datasets, identify a number of significant issues, and select the best candidates for evaluation. (2) We introduce a new evaluation metric for time-series anomaly detection, which—in contrast to previous metrics—is recall consistent and takes temporal correlations into account. (3) We analyze and overhaul existing evaluation protocols and provide the largest benchmark of deep multivariate time-series anomaly detection methods to date. We focus on deep-learning based methods and multivariate data, a common setting in modern anomaly detection. We provide all implementations and analysis tools in a new comprehensive library for Time Series Anomaly Detection, called TimeSeAD.

Submission Length: Regular submission (no more than 12 pages of main content)

Changes Since Last Submission:

- Added author names and affiliations
- Added a link to the official repository
- Added acknowledgements

Code: https://github.com/wagner-d/TimeSeAD

Assigned Action Editor: ~Seungjin_Choi1

License: Creative Commons Attribution 4.0 International (CC BY 4.0)

Submission Number: 735