ZeRO++: Extremely Efficient Collective Communication for Large Model Training

Open Webpage

Guanhua Wang, Heyang Qin, Sam Ade Jacobs, Xiaoxia Wu, Connor Holmes, Zhewei Yao, Samyam Rajbhandari, Olatunji Ruwase, Feng Yan, Lei Yang, Yuxiong He

Published: 01 Jan 2024, Last Modified: 15 May 2025ICLR 2024EveryoneRevisionsBibTeXCC BY-SA 4.0