Pyramid Transformer for Multivariate Time Series Anomaly Detection in IoUT

Yongcan Luo, Ning Chen, Zhanxiang Huang, Jiahao Zheng, Shuxin Qin, Dapeng Oliver Wu

Abstract: Monitoring underwater resources and activities requires the Internet of Underwater Things (IoUT) to rapidly detect anomalies within large volumes of sensory time series data. Despite substantial advancements in deep learning for time series forecasting, representation, and reconstruction, the complex temporal patterns and inherent dependencies in IoUT data streams' stochastic nature still present significant challenges for efficient and rapid feature extraction. In this work, we propose an efficient approach based on the Transformer architecture combined with multi-task learning for anomaly detection in IoUT data streams. Our method employs a pyramid Transformer framework to capture temporal features at multiple resolutions, effectively addressing the stochasticity and temporal dependencies inherent in underwater data. To enhance feature extraction efficiency, we incorporate two down-sampling strategies max pooling and window reduction, within each pyramid layer. These layers utilize a multi-task learning strategy to compute the reconstruction loss. During testing, the results from all layers are integrated to produce the final anomaly detection outcome. Through experiments on four public anomaly detection benchmarks, our method has proven to exceed state-of-the-art levels and also doubles the detection speed compared to previous methods.