DeepSanitizer: Combining Heuristic Rules and Deep Learning Models to Spot Silent Buffer Overflows in Binary

Zhilong Wang, Chen Cao, Li Yu, Suhang Wang, Xiaoyan Sun, Peng Liu

Abstract: Memory sanitizer is a widely employed technique that serves as the frontline in the detection and analysis of memory corruption bugs, such as buffer overflows. Despite its prevalence, existing binary-level sanitizers face challenges in effectively identifying silent buffer overflows – instances where only one or a few adjacent variables are corrupted and usually obvious “symptoms” such crashes are not manifested – due to the complexities of inferring buffer/array boundaries in the absence of source code. This paper introduces DeepSanitizer, a deep-learning-assisted memory sanitizer designed to spot silent buffer overflows. Departing from traditional techniques that attempt to infer data object boundaries, DeepSanitizer relies on abnormal data dependencies and memory access patterns in the overflowed region for buffer overflow detection. By combining heuristic rules and deep learning models, it discerns the distinctions between normal and out-of-bound buffer accesses, predicting whether a buffer segment contains overflows. We implemented a prototype on the Linux system and conducted a series of experiments to assess its effectiveness and performance. Results from our evaluation demonstrate that DeepSanitizer achieves a high accuracy in detecting silent buffer overflows. Additionally, we explored its efficacy in identifying real-world vulnerabilities through experimentation.