Hunting for Offload: Automated Discovery of Acceleratable Code in Datacenters

Joshua Kim, Chaojie Zhang, Íñigo Goiri, Christopher Rossbach, Jovan Stojkovic

Keywords: Accelerators, Agentic AI, Datacenters

TL;DR: Datacenter servers use on-chip accelerators to boost efficiency, but applying them to large codebases is difficult, so we propose an Agentic AI framework that automates their integration and unlocks their full potential.

Abstract:

Modern datacenter CPUs integrate an expanding set of on-chip accelerators for system functions (e.g., compression) and ML inference (e.g., matrix units). Despite their availability, these accelerators see limited adoption in production software, as effective use requires extensive profiling, invasive refactoring, specialized programming models, and hardware expertise—costs that are particularly prohibitive for large, mature codebases. This lack of adoption wastes silicon capability and increases cost and energy consumption.

We argue that agentic AI is a necessary step toward making on-chip accelerators broadly used in datacenter software. However,
accelerator enablement fundamentally differs from conventional CPU-centric optimization: it requires reasoning across heterogeneous and evolving hardware interfaces, handling input-dependent performance regimes, and operating with limited public tooling and training data.

We propose a research agenda that combines agentic-based program understanding with an augmented knowledge base to (1) identify code regions matching accelerator semantics, (2) map them to accelerator-backed implementations, and (3) determine when offloading is beneficial using input-sensitive runtime policies while preserving correctness. By framing accelerator adoption as an iterative, feedback-driven discovery rather than a one-shot compiler transformation, we show how agentic AI can unlock existing hardware capabilities for datacenter software at scale.

Email Sharing: We authorize the sharing of all author emails with Program Chairs.

Data Release: We authorize the release of our submission and author names to the public in the event of acceptance.

Submission Number: 7