Dual-Mode Cloud-Device Collaboration for Efficient Continual Adaptation

Zhu Hongye, Xuan Liu, Jingye Xue, Yanwen Ba, Shigeng Zhang, Kele Xu, Jin Yuan

Keywords: Continual Domain Adaptation, Test-Time Adaptation, Cloud-device Collaboration

Abstract:

Continuous environmental changes induce distribution shifts, leading to significant performance
degradation of models deployed on resource-constrained mobile devices. Existing fast adaptation
methods fail to provide sufficient generalization to meet performance requirements,
while cloud-device collaborative learning often relies on a considerable amount of data,
limiting real-time applicability. To ensure both timeliness and effectiveness, we propose
a dual-mode cloud-device collaborative framework. Specifically, the proposed mothod dynamically
switches modes according to the degree of distribution shift: (1) Collaborative adaptation
mode handles substantial shifts, where the cloud performs multi-level domain alignment and
position-aware prompting to learn domain-invariant representations, which are then distilled
to the device model; (2) Self-adaptation mode addresses minor shifts, where the device model
performs unsupervised test-time adaptation with pseudo-label generation and quality-aware
reweighting for fast local updates. Experimental results show that our framework achieves superior
performance while using only 80\% of the data and incurring less than 0.5\% additional parameters
and computation. Moreover, it consistently outperforms compared methods in both accuracy and
single-frame inference speed.

Primary Area: transfer learning, meta learning, and lifelong learning

Submission Number: 8823