Joint Computation Offloading and Service Caching in Mobile Edge-Cloud Computing via Deep Reinforcement Learning

Ce Shang, Youliang Huang, Yan Sun, Mohsen Guizani

Abstract: Mobile edge computing (MEC) emerges as a promising paradigm that aims to extend the computing capabilities of mobile devices. This enhancement is achieved by offloading heavy computational tasks to nearby edge servers, where relevant services, such as programs, databases, and libraries, are stored to support the execution of these tasks. Given the restricted storage capacity and computing resources of edge servers, it is crucial to carefully consider which tasks to offload and which services to cache. However, the highly dynamic system and closely coupled decisions between offloading and caching pose considerable challenges to developing an effective strategy. In this article, we investigate both computation offloading and service caching in an end-edge–cloud collaborative system under time-varying wireless channels and task requests. We formulate an optimization problem aimed at minimizing the system cost, including time and energy consumption, and model it further as a Markov decision process (MDP). To address this, we propose a deep reinforcement learning (DRL)-based approach with dynamic action masking to concurrently optimize offloading and caching decisions. This approach renormalizes the valid action probability distribution at every step, enabling the agent to explore within the space of valid action combinations and thereby learn the optimal policy. Extensive experimental results demonstrate that the proposed approach greatly reduces the system cost compared to other representative benchmark schemes.