Crucible: Quantifying the Potential of Control Algorithms through LLM Agents

Lianchen Jia, Chaoyang Li, Qian Houde, Tianchi Huang, Jiangchuan Liu, Lifeng Sun

Keywords: Control Algorithms, Large Language Models, Agent, Potential

TL;DR: Quantifying the Potential of Control Algorithms through LLM Agents

Abstract: Control algorithms in production environments typically require domain experts to tune their parameters and logic for specific scenarios. However, existing research predominantly focuses on algorithmic performance under ideal or default configurations, overlooking the critical aspect of Tuning Potential. To bridge this gap, we introduce \texttt{Crucible}, an agent that employs an LLM-driven, multi-level expert simulation to turn algorithms and defines a formalized metric to quantitatively evaluate their Tuning Potential. We demonstrate \texttt{Crucible}'s effectiveness across a wide spectrum of case studies, from classic control tasks to complex computer systems, and validate its findings in a real-world deployment. Our experimental results reveal that \texttt{Crucible} systematically quantifies the tunable space across different algorithms. Furthermore, \texttt{Crucible} provides a new dimension for algorithm analysis and design, which ultimately leads to performance improvements. Our code is available at https://github.com/thu-media/Crucible.

Primary Area: Applications (e.g., vision, language, speech and audio, Creative AI)

Submission Number: 7286