Concurrent Carbon Footprint Reduction (C2FR) Reinforcement Learning Approach for Sustainable Data Center Digital Twin

Soumyendu Sarkar, Avisek Naug, Antonio Guillen, Ricardo Luna Gutierrez, Sahand Ghorbanpour, Sajad Mousavi, Ashwin Ramesh Babu, Vineet Gundecha

Abstract: In recent years, the increasing emphasis on sustainability and carbon footprint reduction has required the exploration of innovative optimization techniques for data center operators. In this paper, we introduce a Concurrent Carbon Footprint Reduction (C2FR) Reinforcement Learning framework, designed to optimize data center energy consumption, load shifting, and battery operation decisions in real time. The C2FR framework utilizes short-term forecasts and incorporates Reinforcement Learning Energy ( <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$A_{E}$</tex> ), Battery ( <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$A_{BAT}$</tex> ) and Load-Shifting ( <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$A_{LS}$</tex> ) agents to optimize and effectively manage the intricate dependencies and information exchange between these individual optimization strategies, thus overcoming the limitations of existing isolated approaches. When compared to state-of-the-art algorithms, the C2FR framework demonstrates its effectiveness across various data center scenarios. The AE agent achieves a 7.9% reduction in pollutant emissions and a 7.8% reduction in energy cost on average. Moreover, the C2FR framework enables further emission reductions through the application of the battery and load-shifting optimization, leading to a total reduction of 10.17% in pollutant emissions on average over different data center configurations. This highlights the potential of the C2FR framework in addressing data center sustainability challenges and improving real-time carbon footprint optimization.