Stream Processing with Adaptive Edge-Enhanced Confidential Computing

Yuqin Yan, Pritish Mishra, Wei Huang, Aastha Mehta, Oana Balmau, David Lie

Abstract: Stream processing is becoming increasingly significant in various scenarios, including security-sensitive sectors. It benefits from keeping data in memory, which exposes large volumes of data in use, thereby emphasising the need for protection. The recent development of confidential computing makes such protection technologically feasible. However, these new hardware-based protection methods incur performance overhead. Our evaluation shows that replacing legacy VMs with confidential VMs to run streaming applications incurs up to 8.5% overhead on the throughput of the queries we tested in the NEXMark benchmark suite. Pursuing specialised protection for broader attacks, such as attacks at the edge with more physical exposure, can push this overhead further. In this paper, we propose a resource scheduling strategy for stream processing applications tailored to the privacy needs of specific application functions. We implement this system model using Apache Flink, a widely-used stream processing framework, making it aware of the underlying cluster's protection capability and scheduling the application functions across resources with different protections tailored to the privacy requirements of an application and the available deployment environment.