BDIP: An Efficient Big Data-Driven Information Processing Framework and Its Application in DDoS Attack Detection

Qiyuan Fan, Xue Li, Puming Wang, Xin Jin, Shaowen Yao, Shengfa Miao, Sizhang Li, Min An, Jing Xu

Abstract: With the rapid advancement of 5G communication technology in the era of big data, massive terminal devices connected to the Internet have dramatically increased the scale of network, generating a large amount of high-dimensional and heterogeneous information. This not only enhances the difficulty of information processing in the network, but also poses a severe challenge to data storage and calculation, which has become a big data problem to be solved urgently. To cope with it, this paper proposes an efficient information processing framework and applies it to Distributed Denial of Service (DDoS) attack detection. Overall, three major highlights are made: (i) Tensor is used to represent multi-modal information in large-scale networks; (ii) A novel denoising algorithm based on tensor train(TT) decomposition is proposed, focused on optimizing both computation and correlation; (iii) A big data-driven information processing framework is developed, which includes information preprocessing, denoising and classification. Results in case study indicate that the framework can achieve an accuracy of 99.19%, all while maintaining the great storage advantage, well speedup ratio and strong computing capabilities under the same computational complexity. It can also be generalized to other network data processing scenarios.