ContraLog: Log File Anomaly Detection with Contrastive Learning and Masked Language Modeling

ICLR 2026 Conference Submission12806 Authors

Keywords: Anomaly detection, Contrastive learning, Masked language modeling, Log file processing

TL;DR: A self-supervised framework that predicts log message embeddings based on context using contrastive learning and masked language modeling.

Abstract: Log files record computational events that reflect system state and behavior, making them a primary source of operational insights in modern computer systems. Automated anomaly detection on logs is therefore critical, yet most established methods rely on log parsers that collapse messages into discrete templates. This discretization discards valuable information. Variable log values are ignored, semantic variation is lost. We propose ContraLog, a parser-free and self-supervised method that reframes log anomaly detection as predicting continuous message embeddings rather than discrete template IDs. ContraLog combines a message encoder that produces rich embeddings for individual log messages with a sequence encoder to model temporal dependencies across sequences. ContraLog is trained with a combination of masked language modeling and contrastive learning to predict masked message embeddings based on the surrounding context. Experiments on the HDFS, BGL, and Thunderbird benchmark datasets empirically demonstrate ContraLogs effectiveness on complex datasets with diverse log messages. Additionally, we find that message embeddings generated by ContraLog carry meaningful information and are predictive of anomalies even without sequence context. These results highlight embedding-level prediction as an approach for log anomaly detection, with potential applicability to other event sequences such as IoT telemetry and audit trails.

Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Submission Number: 12806