Adaptive Confidence Estimation for Data Distribution Shift Robustness in Cloud-Edge Collaborative Inference

Shinan Song, Guangyi Liu, Wenjun Ma, Xiaomao Fan, Jinzhou Cao, Jingyan Jiang

Abstract: Cloud-edge collaborative inference is an effective solution for real-time Internet intelligence applications, with lightweight edge models handling simple samples and offloading difficult ones to the cloud for processing. However, existing methods rely on fixed confidence thresholds or pre-trained routers to distinguish easy and hard samples, but they cannot adapt to data distribution changes from environmental shifts, noise, or new categories. To address these challenges, we propose the Adaptive Confidence Estimation (ACE) algorithm, a novel approach that dynamically adjusts confidence thresholds in real-time to adapt to shifting data distributions. Specifically, ACE improves system robustness by mitigating the impact of distribution shifts, reduces unnecessary offloading to the cloud, and ensures reliable separation of easy and hard inferenced samples. We validate ACE through extensive experiments and show that ACE significantly improves accuracy, reduces latency, and enhances resource utilization compared to existing approaches, providing a scalable and adaptive framework for robust cloud edge collaboration.

External IDs:dblp:conf/adma/SongLMFCJ25