Unsupervised Real Time and Early Anomalies Detection Method for Sewer Networks Systems

Chunming Qiu, Guoxiang Shao, Zhenyu Zhang, Chichun Zhou, Yuejie Hou, Enming Zhao, Xiao Guo, Xiaolin Guan

Abstract:

Sewer networks (SNs) are susceptible to various factors that can lead to failures, resulting in
economic losses and environmental pollution. Data-driven approaches based on sewage flow monitoring
enhance the awareness and maintenance capabilities of SNs. However, the current research lacks early
warning systems for flow anomalies. This presents a challenge for the application of supervised methods,
primarily due to the scarcity of anomalous flow datasets. Even with the availability of such datasets, the
effectiveness of these methods may vary due to environmental differences, since SNs are situated in diverse
environments. Therefore, effectively achieving early warnings for anomalies in unlabeled flow data is a
challenge that must be addressed in the field of flow monitoring. To address this challenge, we propose
a detection method for effectively warning of anomalies in flow data. Since anomalies typically result
in significant deviations from normal data, early warnings can be achieved by comparing the differences
between current and historical data. The key to this early warning lies in establishing an adaptive threshold
for detecting abnormal data changes. Our detection method employs an unsupervised bagging-based multianomaly
detection algorithm to detect such abnormal data changes. Experiments conducted on Erhai Lake
SNs flow data demonstrate that our method can predict anomalies 5-15 minutes in advance with a precision
of 80.00%, a recall of 66.67%, and an F1 score of 0.73. Our approach not only achieves cost-effective and
timely anomalies detection but also overcomes the challenges associated with limited dataset availability,
making it applicable to various other industries.