Cur-CoEdge: Curiosity-Driven Collaborative Request Scheduling in Edge-Cloud Systems

Yunfeng Zhao, Chao Qiu, Xiaoyun Shi, Xiaofei Wang, Dusit Niyato, Victor C. M. Leung

Abstract: The collaboration between clouds and edges unlocks the full potential of edge-cloud systems. Edge-cloud platform has brought about significant decentralization, heterogeneity, complexity, and instability. These characteristics have posed unprecedented challenges to the optimal scheduling problem in the edge-cloud system, including inaccurate decision-making and slow convergence. In this paper, we propose a curiosity-driven collaborative request scheduling scheme in edge-cloud systems, namely Cur-CoEdge. To tackle the challenge of inaccurate decision-making, we introduce a time-scale and decision-level interaction mechanism. This mechanism employs a small-large-time-scale scheduling learning framework, facilitating mutual learning between different decision levels. To address the challenge of slow convergence, we investigate the underlying reasons, such as the sparse reward-setting in reinforcement learning. In response, we develop a curiosity-driven collaborative exploration approach that fosters intrinsic curiosity in the cloud and simultaneously motivates dispatchers to explore the environment both individually and collectively. The effectiveness of this collaborative exploration is also supported by theoretical proof of convergence. Finally, we implement a prototype system on a network hardware system along with two real-world traces. Evaluations demonstrate significant improvements, with up to a 26% increase in time efficiency, a 40% rise in system throughput, and a 71% enhancement in convergence speed.