Algorithmic Heap Layout Manipulation in the Linux KernelDownload PDF


02 May 2022, 12:00 (modified: 23 Jan 2023, 14:38)Submitted to JSYS May 22Readers: Everyone
Keywords: Linux Kernel Exploitation Heap Manipulation
TL;DR: Algorithmic Heap Layout Manipulation in the Linux Kernel to take the art of automating kernel exploits one step further.
Abstract: To evaluate the severity of a security vulnerability a security researcher usually tries to prove its exploitability by writing an actual exploit. In the case of buffer overflows on the heap, a necessary part of this is manipulating the heap layout in a way that creates an exploitable state, usually by placing a vulnerable object adjacent to a target object. This requires manual effort and extensive knowledge of the target. With a target as complex as the Linux kernel, this problem becomes highly non-trivial. At the current time, there has been only little research in terms of employing algorithmic solutions for this. In this work, we present Kernel-SIEVE, a framework for evaluating heap layout manipulation algorithms that target the SLAB/SLUB allocator in the Linux kernel. Inspired by previous work that targets user-space allocators it provides an interface for triggering allocations/deallocations in the kernel and contains a feedback loop that returns the resulting distance of two target objects. With this, we create the (to our knowledge) first performance benchmarks for heap layout manipulation algorithms in the Linux kernel. We present and evaluate two algorithms: A pseudo-random search, which’s performance serves as a baseline, and KEvoHeap, a genetic algorithm based on Heelan’s EvoHeap. We show that KEvoHeap is successful at creating the desired heap layout in all test cases and also surpasses the user-space performance benchmarks of EvoHeap. Finally, we discuss the challenges of applying these kinds of algorithms in real-world scenarios and weigh different possible approaches to tackle the problems that arise.
Area: System Security
Type: Solution
6 Replies