Deep Reinforcement Learning-Based Energy-Conscious Scheduling Under Time-of-Use Electricity Price

Caixia Wang, Wenxuan Fang, Xin Dai, Renchu He, Wei Du, Yang Tang

Abstract: In the context of carbon peaking and carbon neutrality, green production scheduling that considers energy has attracted increasing attention. Reinforcement learning (RL) has emerged as a topic for developing efficient algorithms for solving complex combinatorial optimization problems, such as real-world shop scheduling problems. In this paper, we investigate the energy-conscious scheduling problem (ECSP) under time-of-use (TOU) electricity price with the goal of minimizing both the waiting time and extra electricity cost. A mathematical model is formulated, and an efficient deep RL (DRL)-based optimization method is proposed to solve the problem effectively. We design a novel ECSP network (ECSPNet) tailored to handle various ECSP scales based on the characteristics of the problem. Moreover, the Tchebycheff decomposition method is used to solve the multi-objective optimization problems, complemented by the application of the policy gradient method from reinforcement learning to train the ECSPNet without size limitations. Experiments verify that the proposed ECSPNet outperforms state-of-the-art methods and is computationally efficient, even on instances of larger scales unseen in training. Real-world case studies reveal that the proposed method can reduce annual total electricity cost by approximately 30% while effectively maximizing production efficiency. Note to Practitioners —Enhancing energy consciousness alongside improving production efficiency in manufacturing systems has increasingly become a focal point for both academia and industry, especially with the growing emphasis on environmental concerns and advancing industrialization. Time-of-use (TOU) electricity price policy is widely implemented to effectively balance electricity supply and demand. For business managers, appropriately responding to this policy by optimizing scheduling tasks can significantly reduce energy costs. This paper addresses a novel energy-conscious scheduling problem (ECSP) under TOU electricity price, specifically arising from the electrode graphitization production process in graphite material manufacturing. We propose a deep reinforcement learning-based energy-saving scheduling optimization method, which integrates considerations for both production efficiency and energy cost indicators. By combining the unique characteristics of ECSP with the decision-making capabilities of reinforcement learning and the perception capabilities of deep learning, our method excels in solution speed, efficiency, and adaptability. The effectiveness and practical applicability of our method have been demonstrated through experiments on actual enterprise cases of various scales. The rapidly generated optimization scheduling plans can assist enterprises in rationally arranging scheduling tasks while minimizing electricity cost. Looking ahead, our proposed method has the potential to address a wide range of energy-conscious scheduling problems under TOU electricity price policy.