A Novel Artificial Immune Model on Hadoop for Anomaly Detection

Xinlei Hu, Zhengxia Wang, Yunbing Hu

Abstract: The artificial immune system is an important algorithm for anomaly detection problems. But two major problems of the AIS algorithm are low efficiency of detector generation and excessive number of detectors. In this paper, we propose an improved AIS on Hadoop called AIMAD, which is with antibody suppression. In AIMAD, candidate detectors are generated randomly parallelly in Mapper Jobs, candidate detectors must go through self tolerance. Detectors not only match with self-cell for self tolerance, but also recognize other detectors. In Reducer phase, when a detector is recognized by other detectors, this detector is suppressed. Eliminating those suppressed detectors will significantly reduce detector redundancy in the non-self space. After Reducer phase, candidates detectors will be evolved into mature detectors. The dynamic evolution of the mature detector makes these detectors detect unknown network abnormal behavior or variants of known network abnormal behavior. The theory analysis shows that AIMAD effectively reduces the number of detectors and improves the efficiency of detector generation. Experimental results show that proposed algorithm outperforms other classic algorithms such as RNSA and V-detector. Under three UCI standard datasets, the proposed algorithm can get the highest detection rate, the lowest false alarm rate and the highest time efficiency.