How Does Critical Batch Size Scale in Pre-training?

Open Webpage

Hanlin Zhang, Depen Morwani, Nikhil Vyas, Jingfeng Wu, Difan Zou, Udaya Ghai, Dean P. Foster, Sham M. Kakade

Published: 01 Jan 2025, Last Modified: 15 May 2025ICLR 2025EveryoneRevisionsBibTeXCC BY-SA 4.0