DeepVMUnProtect: Neural Network-Based Recovery of VM-Protected Android Apps for Semantics-Aware Malware Detection

Xin Zhao, Mu Zhang, Xiaopeng Ke, Yu Pan, Yue Duan, Sheng Zhong, Fengyuan Xu

Abstract: The emerging virtual machine-based Android packers render existing unpacking techniques ineffective. The state-of-the-art unpacker falls short because it relies on unreliable heuristics and manually crafted semantic models. Hence, it cannot precisely recover app semantics necessary for malware detection. In this paper, we propose DeepVMUnProtect, a deep learning-based approach to automatically and accurately capture the semantics of VM-packed code, so as to facilitate semantic-based Android malware classification. Experiments have shown that DeepVMUnProtect outperforms the state-of-the-art tool on recovering opcode semantics in Qihoo(58.3%), Baidu(47.5%) and NMMP (58.8%) respectively, and can enable semantics-aware malware detection which prior work fails to do.