Carbon-Neutralizing Edge AI Inference for Data Streams via Model Control and Allowance Trading

Yining Zhang, Lei Jiao, Konglin Zhu, Yuedong Xu, Lin Zhang

Abstract: To make edge AI inference carbon-neutral, we perform a comprehensive mathematical and algorithmic study on the complex online management of AI model selection and placement with carbon allowance trading. This work is non-trivial due to the critical challenges such as the unknown stochastic distributions and arrivals of inference data, the exploration-exploitation tradeoff with model switching cost, and the uncertain, time-varying allowance prices and system environments. We first model a long-term stochastic cost optimization problem to capture these challenges. Then, we design a novel learning-centric decomposition-based online algorithmic framework which, on the one hand, samples and places the models repeatedly to minimize the expected inference loss with bounded model switches, and on the other hand, buys and sells carbon allowances cost-efficiently in real time toward carbon neutrality without relying on future allowance prices and system emissions. We further formally prove multiple performance guarantees of our algorithms in terms of sub-linear regret and fit. Finally, we conduct trace-driven evaluations to confirm the substantial advantages of our approach compared to baselines and state-of-the-arts in practice.

External IDs:dblp:conf/icdcs/Zhang0Z0025