Enhancing Anomaly Detection with Entropy Regularization in Autoencoder-based Lightweight Compression

Andriy Enttsel, Alex Marchioni, Gianluca Setti, Mauro Mangia, Riccardo Rovatti

Abstract: Monitoring systems produce and transmit large amounts of data. For an efficient transmission, data is often compressed and autoencoders are a widely adopted neural network-based solution. However, this processing step leads to a loss of information that may negatively impact the performance of downstream tasks, such as anomaly detection. In this work, we propose a loss function for an autoencoder that addresses both compression and anomaly detection. Our key contribution is the inclusion of a regularization term based on information-theoretic quantities that characterize an anomaly detector processing compressed signals. As a result, the proposed approach allows for a better use of the communication channel such that the information preserved by the compressed signal is optimized for both detection and reconstruction, even in scenarios with lightweight compression. We tested the proposed technique with ECG signals affected by synthetic anomalies and the experiments demonstrated an average 17% increase in the probability of detection across three standard detectors. Additionally, we proved that our approach is generalizable to image data.