Energy-Efficient Analytics for Geographically Distributed Big Data

Peng Zhao, Xinyu Yang, Jie Lin, Shusen Yang, Wei Yu

Abstract: Big data analytics on geographically distributed datasets (across data centers or clusters) has been attracting increased interest in both academia and industry, posing significant complications for system and algorithm design. In this paper, we systematically investigate the geodistributed big data analytics framework by analyzing the fine-grained paradigm and key design principles. We present a dynamic global manager selection algorithm to minimize energy consumption cost by fully exploiting the system diversities in geography and variation over time. The algorithm makes real-time decisions based on measurable system parameters through stochastic optimization methods, while achieving performance balance between energy cost and latency. Extensive trace-driven simulations verify the effectiveness and efficiency of the proposed algorithm. We also highlight several potential research directions that remain open and require future elaborations in analyzing geodistributed big data.