Cloud-Edge Collaborative Large Model Services: Challenges and Solutions

Yanghe Pan, Zhou Su, Yuntao Wang, Shaolong Guo, Han Liu, Ruidong Li, Yuan Wu

Abstract: The rapid development of large models such as GPT-4 and Midjourney has spawned worldwide attention in various fields. To practically deploy large model services in downstream tasks, cloud-edge collaboration mechanism offers an appealing solution by seamlessly sharing fresh data, knowledge, and resources between the cloud and distributed edge nodes. Offloading the fine-tuning process of large models to edge networks through federated learning (FL) to implement efficient and privacy-preserving large model services has become an emerging research direction. However, there exist challenges in terms of privacy leakages, data heterogeneity, and edge resource constraints in delivering cloud-edge collaborative large model services. In this paper, we propose an innovative cloud-edge collaborative framework for federated large model training and deployment with reduced communication overheads and enhanced scalability. Within this framework, we further devise a data-model-driven privacy protection mechanism, a lightweight reciprocal knowledge transfer approach, and an optimal and reliable edge node recruitment strategy to fully harness edge data/knowledge/resources with enhanced privacy protection under high heterogeneity. Through an experimental case study, we validate the effectiveness of the proposed framework and solutions. Finally, we outline significant future research directions in this evolving field.

External IDs:dblp:journals/network/PanSWGLLW25