Medusa: Unveil Memory Exhaustion DoS Vulnerabilities in Protocol Implementations

Zhengjie Du, Yuekang Li, Yaowen Zheng, Xiaohan Zhang, Cen Zhang, Yi Liu, Sheikh Mahbub Habib, Xinghua Li, Linzhang Wang, Yang Liu, Bing Mao

Keywords: Memory exhaustion, DoS vulnerabilities, Protocol

Abstract:

Web services have brought great convenience to our daily lives. Meanwhile, they are vulnerable to Denial-of-Service (DoS) attacks.
DoS attacks launched via vulnerabilities in the services can cause great harm. The vulnerabilities in protocol implementations are
especially important because they are the keystones of web services. One vulnerable protocol implementation can affect all the web
services built on top of it. Compared to the vulnerabilities that cause the target service to crash, resource exhaustion vulnerabilities are
equally if not more important. This is because such vulnerabilities can deplete the system resources, leading to the unavailability of not
only the vulnerable service but also other services running on the same machine. Despite the significance of this type of vulnerability,
there has been limited research in this area.

In this paper, we propose Medusa, a dynamic analysis framework to detect memory exhaustion vulnerabilities in protocol implementations, which are the most common type of resource exhaustion vulnerabilities. Medusa works in two phases: exploration phase and verification. In the exploration phase, a protocol property graph (PPG) is constructed to embed the states with relevant properties including memory consumption information. In the verification phase, the PPG is used to simulate DoS attacks to verify the vulnerabilities. We implemented Medusa and evaluated its performance on 21 implementations of five protocols. The results demonstrate that Medusa outperforms the state-of-the-art techniques by discovering overall 127× maximum memory consumption. Lastly, Medusa has discovered six 0-day vulnerabilities in six protocol implementations for three protocols. Particularly, one of the vulnerabilities was found in Eclipse Mosquitto, which can affect thousands of services and it has been assigned with a CVE ID.

Track: Security

Submission Guidelines Scope: Yes

Submission Guidelines Blind: Yes

Submission Guidelines Format: Yes

Submission Guidelines Limit: Yes

Submission Guidelines Authorship: Yes

Student Author: Yes

Submission Number: 1008