DeMAAE: deep multiplicative attention-based autoencoder for identification of peculiarities in video sequences

Nazia Aslam, Maheshkumar H. Kolekar

Abstract: In videos, anomaly detection is challenging due to its diverse nature in different application domains. Reconstruction and prediction-based methods have been widely employed to detect anomalies. Due to the generalization capability of a deep neural network, sometimes, it recreates irregular patterns along with regular ones. This paper presents a novel autoencoder-based framework called deep multiplicative attention-based autoencoder (DeMAAE) to detect anomalies in a video sequence. The global attention mechanism is used at the decoder side of DeMAAE for better feature learning during the decoding phase. An attention map is created by taking the dot product between all encoder’s hidden states and the previously generated decoder’s hidden state. After that, the final output of the decoder is determined by the context vector. The context vector is computed using the weighted summation of all encoder’s hidden states and attention weight. DeMAAE delivers an improved runtime of 0.015 s (\( \sim \) 67 fps) for detecting anomalies during testing. Extensive experiments have been performed on the two diversified and widely used datasets (UCSD Pedestrian and CUHK Avenue) to compare the efficacy of DeMAAE with different state-of-the-art methods.

External IDs:doi:10.1007/s00371-023-02882-2