NeVe: A Log-based Fast Incremental Network Feature Embedding Approach

Ran Tian, Zulong Diao, Haiyang Jiang, Gaogang Xie

Abstract: Similarity (distance) measurement among network features (e.g. IP address, MAC address, port number, and protocol, etc.) based on network logs is a critical step for data mining in intrusion detection, anomaly prediction, and log analysis. A practical approach is necessarily accurate, fast, and incremental due to the dynamic network environment. However, existing solutions fail to satisfy these demands simultaneously. Therefore, we propose a novel unsupervised network feature embedding approach: Network Vector (NeVe). It learns the similarity from context information by introducing a natural language processing algorithm GloVe. Since the network data is more timeliness with an almost infinite corpus size, we adjust the algorithm to adapt the input data format and design a fast scalable online update mechanism. Our evaluation demonstrates that NeVe can achieve the highest accuracy with minimal time consumption (13 ~ 15 times faster) compared with the state-of-the-art approach.