Der: Unifying DVFS and Early-Exit for Embedded AI Inference via Reinforcement Learning

Yuting He, Jingjin Li, Chengtai Li, Qingyu Yang, Zheng Wang, Heshan Du, Jianfeng Ren, Heng Yu

Abstract: Executing neural networks on resource-constrained embedded devices faces challenges. Efforts have been made at the application and system levels to reduce the execution cost. Among them, the early-exit networks reduce computational cost through intermediate exits, while Dynamic Voltage and Frequency Scaling (DVFS) offers system energy reduction. Existing works strive to unify early-exit and DVFS for combined benefits on both timing and energy flexibility, yet limitations exist: 1) varying time constraints that make different exit points become more, or less, important in terms of inference accuracy, are not taken care of, and 2) the optimal decisions of unifying DVFS and early-exit as a multi-objective optimization problem are not achieved due to the large configuration space. To address these challenges, we propose De2r, a reinforcement learning-based framework that jointly optimizes early-exit points and DVFS settings for continuous inference. In particular, De2r includes a cross-training mechanism that fine-tunes the early-exit network to accommodate dynamic time constraints and system conditions. Experimental results demonstrate that De2r achieves up to 22.03% energy reduction and 3.23% accuracy gain compared to contemporary techniques.

External IDs:dblp:conf/date/HeLLYWDR025