DAG-aware harmonizing job scheduling and data caching for disaggregated analytics frameworks

Yulai Tong, Jiazhen Liu, Hua Wang, Mingjian He, Ke Zhou, Rongfeng He, Qin Zhang, Cheng Wang

Abstract: Highlights•A coordinated scheduling and caching system for modern analytics frameworks.•A cache-driven scheduling algorithm to enable more effective caching.•A scheduling-aware caching strategy to best utilize the cache-driven execution.•Proposing a detailed implementation that integrates Tripod with Tez.