A Large Deviations Approach to Statistical Traffic Anomaly Detection

Ioannis Ch. Paschalidis, Georgios Smaragdakis

Abstract: We introduce an Internet traffic anomaly detection mechanism based on large deviations asymptotic results. Using past traffic traces we characterize network traffic during various time-of-day intervals, assuming that it is anomaly-free. We present two different approaches to characterize traffic: (i) a model-free approach based on the method of types and Sanov's theorem, and (ii) a model-based approach modeling traffic using a Markov modulated process. Using these characterizations as a reference we continuously monitor traffic and employ large deviations results to compute the probability that the monitored traffic is "consistent" with the corresponding reference characterization. Low values of this probability identify, in real-time, traffic anomalies. Our experimental results show that applying our methodology (even short-lived) anomalies are identified within a small number of observations. Throughout, we compare the two approaches presenting their advantages and disadvantages. We validate our techniques by analyzing real traffic traces with time-stamped anomalies