DIDS: A distributed inference framework with dynamic scheduling capability

Yuwei Yan, Yikun Hu, Qinyun Cai, Wangdong Yang, Kenli Li

Abstract: Highlights•Static inference cannot adapt to the dynamic changes in the edge environment.•Two basic scheduling strategies: Re-Partition and Complete-Push.•Dynamic Push is the proposed 3-stages runtime scheduler.•DIDS achieves a noticeable speedup compared with static distributed inference and state-of-the-art dynamic distributed inference with real-world settings.