Enhanced Q-learning algorithm for dynamic power management with performance constraint

Wei Liu, Ying Tan, Qinru Qiu

Abstract: This paper presents a novel power management techniques based on enhanced Q-learning algorithms. By exploiting the sub modularity and monotonic structure in the cost function of a power management system, the enhanced Q-learning algorithm is capable of exploring ideal trade-offs in the power-performance design space and converging to a better power management policy. We further propose a linear adaption algorithm that adapts the Lagrangian multiplier ¿ to search for the power management policy that minimizes the power consumption while delivering the exact required performance. Experimental results show that, comparing to the existing expert-based power management, the proposed Q-learning based power management achieves up to 30% and 60% reduction in power saving for synthetic workload and real workload, respectively while in average maintain a performance within 7% variation of the given constraint.