AC Loop: An LLM-Based End-to-End Auto Calibration Framework

Hao Zhen, Qingxuan Kang, Yungang Bao, Trevor E. Carlson

Keywords: LLM, microbenchmarks, modeling

TL;DR: We introduce AC Loop, an LLM- driven end-to-end automated calibration framework.

Abstract:

High-level architectural simulators are widely used for performance exploration during hardware design.
However, performance differences from the RTL design can lead to inaccurate estimates.
Existing approaches address this through model calibration to ensure faithful reflection of the target hardware.
One recent calibration methodology proposes single-feature-attribution microbenchmarks as a way to target microarchitecture structures.
But, these microbenchmarks are manually designed requiring substantial engineering effort and remaining prone to errors.

To improve calibration efficiency, we introduce \textit{AC Loop}, an LLM-driven end-to-end automated calibration framework.
The AC loop leverages a modern LLM to generate calibration-oriented microbenchmarks.
Given sufficient background information and structured guidance, the LLM can effectively produce microbenchmarks that enable rapid calibration.
AC loop provides a fully automated, end-to-end pipeline that integrates calibration, microbenchmark execution, and result analysis end to end.

We evaluate AC Loop on the RTL and gem5 modules of the OpenXiangShan platform. The results show that it effectively generates microbenchmarks, corrects misaligned parameters, and detects fine-grained discrepancies.
In one case study, the AC Loop took just 7.7 minutes to automatically generate appropriate microbenchmarks for characterizing Load Queue capacity, consuming approximately ten thousand input and output tokens.

Email Sharing: We authorize the sharing of all author emails with Program Chairs.

Data Release: We authorize the release of our submission and author names to the public in the event of acceptance.

Submission Number: 14