Online Learning for Right-Sizing Serverless Functions

Prasoon Sinha, Kostis Kaffes, Neeraja J. Yadwadkar

Keywords: machine learning for systems, serverless computing, resource allocation and management

TL;DR: This paper presents a system that makes fine-grained and decoupled resource allocations per-invocation using an online learning agent to improve user performance and cost while decreasing resource underutilization.

Abstract:

Serverless computing relieves developers from the burden of allocating and managing resources for their cloud applications, providing ease-of-use to the users and the opportunity to optimize resource utilization to the providers. However, the lack of visibility into user functions limits providers’ ability to right-size the functions. Thus, providers resort to simplifying assumptions, ignoring input variability, and coupling different resource types (CPU, memory, network), resulting in widely varying function performance and resource efficiency. To provide users with predictable performance and costs for their function executions, we need to understand how these factors contribute to function performance and resource usage.
In this paper, we first conduct a deep study of commonly deployed serverless functions on an open-source serverless computing framework. Our analysis provides key insights to guide the design of a resource allocation framework for serverless systems, including the need to provision resources per invocation, account for function semantics, and decouple resources. We then present Lachesis, a resource allocation framework that builds on the insights we found and leverages online learning to right-size a function invocation. Our experiments show that Lachesis can increase speedup by 2.6x while decreasing idle cores by 82% compared to static allocation decisions made by users.

Workshop Track: MLArchSys

Presentation: In-Person

Presenter Full Name: Prasoon Sinha

Presenter Email: [e-mail]

Presenter Bio: Prasoon is a Ph.D. student in the ECE Department at The University of Texas at Austin advised by Professor Neeraja J. Yadwadkar. His broad research interests are in computer systems, cloud computing, and the boundary between systems and machine learning.