An Efficient Data Collection Module on Petascale Systems

Huang Huang, Tong Xiao, Chuanying Li, Can Leng, Yutong Lu

Abstract: Not only for increasing scalability up to exascale systems but even for contemporary supercomputer architectures does it require substantial efforts to efficiently collect that contains system fault information within the framework of faults prediction. In this term, the article mainly focuses on efficient data collection, preferring to optimize an effective module to improve the efficiency of data collection in petascale system. The core of our module includes a data collection acceleration layer scheduled by H2FS. Hereafter, we conducted extensive tests based on a petascale system to verify the solution, and the experimental results demonstrate to be effectiveness and scalability of our module.