One-Class Self-Attention Model for Anomaly Detection in Manufacturing Lines

Linh Le, Srivatsa Mallapragada, Shashank Hebbar, David A. Guerra-Zubiaga

Abstract: In this paper, we present our case study on anomaly detection in manufacturing lines. More specifically, our goal is to explore machine learning and deep learning models, including our designed architectures, on detecting different types of irregularities in data that is collected from a manufacturing system in operations. We focus on four types of sensors which measure air pressures and water flows of a liquid injection process, and positions and torques of a transportation motor. The system works in a cyclic nature - the collected data can be divided into cycles with similar patterns, each of which form a data instance in this anomaly detection task. Since procuring labeled data with actual anomalies is costly, we simulate four types of abnormal patterns to examine the models’ behaviors in each case. The tested models include One-Class Support Vector Machine, Isolation Forest, Deep Auto-Encoders, and Deep One-Class Classifier. We further empirically design two deep learning architectures that are called One-Class Self-Attention (OCSA) models. OCSA integrates self-attention mechanisms with the one-class classifier training objective to incorporate the representation capacity of the former and the modeling capability of the latter. Our experimental study shows that our proposed designs consistently achieve the highest or competitive performances in both detection rates and in running times in a majority of tests.