TransLog: A Unified Transformer-based Framework for Log Anomaly Detection

Hongcheng Guo, Jian Yang, Jiaheng Liu, Jiaqi Bai, Yan Ling, Zhoujun Li, Tieqiao Zheng, Liangfan Zheng, Weihao Hou, Bo Zhang

Keywords: System Security, Log data, Anomaly detection

Abstract: Log anomaly detection is a key component in the field of artificial intelligence for IT operations (AIOps). Considering the log data of variant domains, retraining the whole network for unknown domains is inefficient in real industrial scenarios, especially for low-resource domains. However, previous deep models merely focused on extracting the semantics of log sequences in the same domain, leading to poor generalization on multi-domain logs. To alleviate this issue, we propose a unified Transformer-based framework for Log anomaly detection (TransLog) to improve the generalization ability across different domains from a new perspective, where we establish a two-stage process including the pre-training and adapter-based tuning stage. Specifically, our model is first pre-trained on the source domain to obtain shared semantic knowledge of log data. Then, we transfer such knowledge to the target domain via shared parameters. Besides, The adapter, designed for log data, is utilized to improve migration efficiency while reducing cost. The proposed method is evaluated on three public datasets and one real-world dataset. Experimental results demonstrate that our simple yet efficient approach, with fewer trainable parameters and lower training costs in the target domain, achieves state-of-the-art performance on all benchmarks.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Submission Guidelines: Yes

Please Choose The Closest Area That Your Submission Falls Into: Applications (eg, speech processing, computer vision, NLP)