Anomaly Detection in Network Traffic Using Advanced Machine Learning Techniques

Stephanie Ness, Vishwanath Eswarakrishnan, Harish Sridharan, Varun Shinde, Naga Venkata Prasad Janapareddy, Vineet Dhanawat

Abstract: Anomaly detection in network traffic is a critical aspect of network security, particularly in defending against the increasing sophistication of cyber threats. This study investigates the application of various machine learning models for detecting anomalies in network traffic, specifically focusing on their effectiveness in addressing challenges such as class imbalance and feature complexity. The models assessed include Isolation Forest, Naive Bayes, XGBoost, LightGBM, and SVM classification. Through comprehensive evaluation, this research explores both supervised and unsupervised approaches, comparing their performance across key metrics like accuracy, F1-score, and recall. The results reveal that while models like XGBoost and LightGBM exhibit impressive performance, with LightGBM achieving near-perfect training accuracy (1.0) and solid test accuracy (0.85), others like Isolation Forest show limitations with low accuracy. The study highlights the strengths and weaknesses of each model, providing valuable insights into their practical application for network anomaly detection. By comparing different algorithms, this research contributes to advancing the application of machine learning in network security, offering guidance on model selection and optimization for improved detection of cyber threats.