Network anomaly detection based on TCM-KNN algorithm

Yang Li, Binxing Fang, Li Guo, You Chen

Abstract: Intrusion detection is a critical component of secure information systems. Network anomaly detection has been an active and difficult research topic in the field of Intrusion Detection for many years. However, it still has some problems unresolved. They include high false alarm rate, difficulties in obtaining exactly clean data for the modeling of normal patterns and the deterioration of detection rate because of some "noisy" data in the training set. In this paper, we propose a novel network anomaly detection method based on improved TCM-KNN (Transductive Confidence Machines for K-Nearest Neighbors) machine learning algorithm. A series of experimental results on the well-known KDD Cup 1999 dataset demonstrate it can effectively detect anomalies with high true positive rate, low false positive rate and high confidence than the state-of-the-art anomaly detection methods. In addition, even interfered by "noisy" data (unclean data), the proposed method is robust and effective. Moreover, it still retains good detection performance after employing feature selection aiming at avoiding the "curse of dimensionality".