Caching on the Sky: A Multiagent Federated Reinforcement Learning Approach for UAV-Assisted Edge Caching

Xuanheng Li, Jiahong Liu, Xianhao Chen, Jie Wang, Miao Pan

Abstract: As a promising solution to alleviate network congestion, mobile edge caching based on unmanned aerial vehicles (UAVs) has emerged and received intensive research interests, where users could download their desired contents from UAVs with much lower latency. As for the UAV-assisted edge caching, to improve the users’ Quality of Experience while reducing the cost on content updating, how to jointly design the trajectory and caching strategy for UAVs is critical. However, considering the dynamics and uncertainty on the traffic environment, as well as the mutual effect among different UAVs, such joint design is nontrivial. In this article, we propose a collaborative joint trajectory and caching scheme for UAV-assisted networks under the dynamic and uncertain traffic environment. Unlike most existing work relying on model-based or single-agent methods, we develop a multiagent deep reinforcement learning (MADRL) approach to obtain the solution, where the specific content demand model is not needed and each UAV would learn the best decision autonomously based on its local observations. It can achieve the adaptive cooperation among different UAVs, while optimizing the overall network performance. Moreover, standing from the perspective on swarm intelligence, we further develop a dynamic clustering federated learning framework on the MADRL algorithm. By performing parameter fusion, each UAV can improve the learning efficiency.