Online HVAC Optimization under Comfort Constraints via Reinforcement Learning

Christian Stippel, Rafael Sterzinger, David Sengl, Aleksey Bratukhin, Markus Kobelrausch, Stefan Wilker, Thilo Sauter

Abstract: This paper shows the capabilities of Reinforcement Learning to enhance the efficiency of heating, ventilation, and air conditioning systems within office buildings. Our research applies the precise management of temperature and humidity, fundamental control algorithms, and several other factors to reduce the building’s power consumption while improving thermal comfort and air quality. We succeed in developing optimal control policies by employing Proximal Policy Optimization and Advantage Actor Critic. The outcomes of our research indicate that our RL framework substantially outperforms existing baselines in maintaining ideal humidity and temperature levels while achieving a notable reduction in energy consumption by $12 \%$ over seven years compared to the current static control logic employed in HVAC systems. The contributions of our research include introducing RL agents trained online for effective and economical HVAC control from day one and an underlying shared state embedding space to effectively understand the dynamics between various rooms. We compare our approach against four baseline control logics. Moreover, we show a novel socket communication protocol to seamlessly interact with TRNSYS18, a simulation environment that enables rapid training and evaluation of our agents.