Online Anomaly Detection for Streaming Data in the Presence of Missing Values

Xinyao Xu, Mengna Liu, Xu Cheng, Lei Song, Jianhua Zhang

Abstract: Online anomaly detection is a critical area in data analysis, particularly for handling dynamic data streams and addressing the challenge of concept drift. While current methods for online anomaly detection have achieved significant breakthroughs, creating a system that can continuously and effectively learn in scenarios with missing data remains a formidable challenge. In this paper, we introduce an autoencoder-based online deep anomaly detection model that addresses both missing data and concept drift. The model features a lightweight module specifically designed for efficient missing value processing. Additionally, it incorporates an adaptive model pool to manage the time-varying concept drift commonly observed in dynamic data streams. This flexible and dynamic management mechanism allows the model to adapt to changes in the data stream, maintaining robust anomaly detection performance across various conditions. Empirical validation of our model through ten comparative experiments on high-dimensional datasets affected by concept drift shows that it outperforms existing state-of-the-art methods. These results underscore the effectiveness and practicality of our approach.