Data-Centric and Heterogeneity-Adaptive Sequence Parallelism for Efficient LLM Training.

Yujie Wang, Shiju Wang, Shenhan Zhu, Fangcheng Fu, Xinyi Liu, Xuefeng Xiao 0001, Huixia Li, Jiashi Li, Faming Wu, Bin Cui 0001