AcLog: An Approach to Detecting Anomalies from System Logs with Active Learning

Chiming Duan, Tong Jia, Ying Li, Gang Huang

Abstract: Log-based anomaly detection is an essential aspect of maintaining software reliability, particularly in the context of microservice systems. However, existing log-based anomaly detection approaches rely on historical anomalous labeled data or require huge labeling efforts. This makes existing log-based anomaly detection approaches inefficient. In this paper, we propose AcLog, a novel anomaly detection approach that incorporates human knowledge to enhance model ability based on the framework of deep active learning. It incorporates an unsupervised model to learn from normal log data rather than historical anomalous labeled data and leverages active learning to incorporate human knowledge as a golden signal to augment the quality of training log data. Experiment results on three open log datasets and one log dataset collected from a real-world microservice system show that our approach improves over 7% F1-score with 5% labeled training data on average.