Breaking Static Paradigms: A Mutual Evolution Framework for Edge-Cloud Model Collaboration

ICLR 2026 Conference Submission19641 Authors

Keywords: Edge-Cloud Model Collaboration, Mutual Evolution, Large Language Models

Abstract:

To simultaneously achieve high performance and low latency, the paradigm of edge-cloud model collaboration, where Large Language Models (LLMs) are deployed on the powerful cloud and Small Language Models (SLMs) on the resource-limited edge devices, has garnered great attention recently. However, a key limitation of current edge-cloud architecture is its static nature, which hinders
the dynamic integration of new knowledge. More specifically, existing methods typically update the system by directly retraining the cloud-based LLM with edge-side newly collected data, which not only increases communication overhead but also neglects available computing power and data accessibility on edge devices. To tackle this challenge, we propose a novel mutual Evolution framework for edge-cloud model Collaboration called CoEvo that enables both cloud-side LLM and edge-side SLMs to update with new knowledge continuously. The cloud-based LLM can enhance edge-side SLMs through credible Chain-of-Thought (CoT) based knowledge distillation to improve its general understanding capabilities. Once the edge-side SLMs collect new domain-specific knowledge and optimize themselves locally, they will specifically enhance the cloud-based LLM via a credible probability matrix predicted on a few samples without uploading all raw data. Through this mutual evolution, the system can achieve continual optimization of the cloud and edge-side models and promote real-world deployments. Experimental results demonstrate a considerable performance gain of our edge-side SLMs against existing methods on the target dataset, with the cloud-side LLM also achieving a notable improvement over the base model.

Primary Area: foundation or frontier models, including LLMs

Submission Number: 19641