Mining Open-Source Cyber Threat Intelligence with Distant Supervision from the Web

Panpan Zhang, Jing Ya, Tingwen Liu, Jinqiao Shi

Abstract: Mining open-source cyber threat intelligence plays an important role for cybersecurity professionals in quickly understanding cyber threats and make preventive measures timely. Much work has been proposed, however, they utilize fixed-point monitoring strategy, and have the limitations in coverage, cost and accuracy. To overcome these limitations, we propose a novel approach of mining open-source cyber threat intelligence with distant supervision from the Web. The approach can cover as many open-source data feeds as possible with the help of search engines. We analyze the retrieval results from search engines to determine whether the search targets belong to cyber threat intelligence or not. Specifically, we reduce it to a multiple instance binary classification problem, and use multiple instance neural networks to achieve more accurate classification. Experimental results on a real-world dataset show the significant effectiveness of our work.