Lightweight AI for Efficient Resource Management in Heterogeneous-Core Architectures

Joshua Kim, Chaojie Zhang, Íñigo Goiri, Christopher Rossbach, Jovan Stojkovic

Keywords: Datacenters, Phase Prediction, Microservices

TL;DR: Datacenter apps rapidly shift among fine-grained phases, making static tuning ineffective; we use in-hardware AI phase prediction to steer threads across specialized chiplets for better perf/W.

Abstract:

Datacenter applications increasingly exhibit fine-grained workload heterogeneity, driven by microservice architectures, datacenter ``tax'' operations, and diverse intra-service components.
These factors lead to rapidly shifting execution phases with distinct and short-lived resource demands, which are difficult for static or hand-tuned resource management techniques to capture.
In this paper, we present lightweight AI techniques for phase-aware resource management in heterogeneous-core server architectures.
We characterize phase-level heterogeneity across representative datacenter workloads and show that simple threshold-based predictors struggle under fine-grained and noisy execution behavior. We then evaluate several low-cost machine learning approaches for online phase prediction and identify random forests as a promising design point, balancing accuracy, robustness, and inference overhead.
We leverage the phase predictor to design a heterogeneous-core server architecture to improve Performance/Watt in datacenters while maintaining application transparency and ease of programming.

Email Sharing: We authorize the sharing of all author emails with Program Chairs.

Data Release: We authorize the release of our submission and author names to the public in the event of acceptance.

Submission Number: 8