Detecting Trojan horses based on system behavior using machine learning method

Yu-Feng Liu, Liwei Zhang, Jian Liang, Sheng Qu, Zhiqiang Ni

Abstract: The Research of detection malware using machine learning method attracts much attention recent years. However, most of research focused on code analysis which is signature-based or analysis of system call sequence in Linux environment. Obviously, all methods have their strengths and weaknesses. In this paper, we concentrate on detection Trojan horse by operation system information in Windows environment using data mining technology. Our main content and contribution contains as follows: First, we collect Trojan horse samples in true network environment and classify them by scanner. Secondly, we collect operation system behavior features under infected and clean circumstances separately by WMI manager tools. And then, several classic classification algorithms are applied and a performance comparison is given. Feature selection methods are applied to those features and we get a feature order list which reflects the relevance order of Trojan horse activities and the system feature. We believe the instructive meaning of the list is significant. Finally, a feature combination method is applied and features belongs different groups are combined according their characteristic for high classification performance. Results of experiments demonstrate the feasibility of our assumption that detecting Trojan horses by system behavior information is feasible and affective.