Strategies for Fast I/O Throughput in Large-Scale Climate Modeling Applications

Koushik Sen, Sathish Vadhiyar, P. N. Vinayachandran

Abstract: Large-scale HPC applications are highly data-intensive with significant times spent in I/O operations. Many large-scale scientific applications do not adequately optimize the I/O operations, leading to overall poor performance. In this work, we have developed two main strategies for providing fast I/O throughput for an important climate modeling application, namely, Regional Ocean Modeling System (ROMS) that uses NetCDF for I/O operations. The strategies include load balancing the I/O operations and selective writing of data. We have also implemented file striping to improve I/O performance. Our experiments with up to 1440 processor cores and 5 days of simulations showed that our load balancing strategy resulted in about 27 % decrease in execution times over the default executions, our selective writing strategy resulted in a further decrease of about 30 % and the optimized file striping resulted in a further decrease of about 12 % in execution times. All the strategies combined together improved the overall performance of the application by about 70 %.