Collaborative Reinforcement Learning Based Route Planning for Cloud Content Delivery Networks

Mingxin He, Dianjie Lu, Jie Tian, Guijuan Zhang

Abstract: Cloud Content Delivery Networks (CCDNs) scale their content delivery services by taking advantage of the flexible cloud resources and management mechanisms so that users can get the content nearby and reduce traffic of the central network. The content delivery efficiency of CCDNs and quality of service can be improved by rationally planning routing. However, the existing routing planning strategies are not suitable for CCDNs because the number of nodes and the network traffic in CCDNs are dynamic. In this paper, we propose a routing planning method that considers the influence of these factors. In our method, we first build a new framework called the collaborative reinforcement learning based CCDN to capture the dynamic characteristics of the above factors through the V -value feedback model, and we quantify the node transmission efficiency with the Q value. Based on this framework, we propose a collaborative reinforcement learning based delivery tree construction algorithm to obtain a delivery tree. Through numerical simulation, it is found that the delivery path constructed by this method can effectively improve the efficiency of content delivery.