Towards Adversarially Robust Human-in-the-Loop Learning for HVAC Systems
Keywords: Human in the loop, Network enabled AI, Network enabled crowdsourcing, HVAC systems, Adversarial Reinforcement Learning
Abstract: Reducing global energy consumption is an urgent step towards slowing the pace of global warming and climate change. Many opportunities to do so lie in the building sector, which is a major energy occupant, particularly Heating, Ventilation and Air-Conditioning (HVAC) systems. Centralized air conditioning in large communal spaces (e.g. malls, offices, cinemas, libraries) often aims to cool spaces to very low temperatures, leading to significant energy consumption that may not even be necessary for users' comfort.
To address this, human-in-the-loop learning (HIL), a network-enabled crowdsourced reinforcement learning (RL) framework has been proposed. This framework leverages direct thermal comfort feedback from occupants to optimize energy efficiency and thermal comfort in HVAC systems in public buildings. Nevertheless, in HIL, control systems may receive unreliable feedback from adversarial or irrational users. Therefore, in this work we work towards increasing the safety and robustness of HIL frameworks in HVAC systems. We propose RARL_HIL, in which a primary agent is jointly trained with an adversarial agent which aims to destabilize the system via generating 'false' feedback. The primary agent learns to operate effectively in challenging and destabilizing environments. Simulation results shows that our algorithm outperforms a traditional human in the loop RL algorithm, in unseen test environments involving adversarial or irrational user feedback.
Submission Number: 15
Loading