Automated Threat-Alert Screening for Battling Alert Fatigue with Temporal Isolation Forest

Muhamad Erza Aminanto, Lei Zhu, Tao Ban, Ryoichi Isawa, Takeshi Takahashi, Daisuke Inoue

Abstract: Network-based intrusion detection systems (NIDSes) tend to output massive alert logs to cover all suspicious communications that deviate from normal network traffic. Due to the tremendous volume of these alert logs, real-time incident response or keeping in pace with the alerts sometimes turns out to be impractical for security operators who have to genuinely investigate each alert to verify whether immediate remedial action is necessary. This problem, known as the threat-alert fatigue problem, causes many unexplored alerts and hence deteriorates the quality of service (QoS). In order to reduce the massive number of alerts, we propose an alert screening scheme that can triage alerts on the basis of the potential of a vast threat. We leverage the fully unsupervised nature of the adopted isolation forest method. Our proposed scheme does not require any prior labeling information and is thus suitable for most NIDSes deployed in enterprise environments. Moreover, by taking advantage of the temporal information in the alerts, we observe that each period (currently set to one day) has its distinct characteristics, which can be exploited to isolate anomalies. This study demonstrates the advantages of unsupervised learning in reducing vast threat alerts and lays the groundwork for battling the alert fatigue problem.