Straight to Zero: Why Linearly Decaying the Learning Rate to Zero Works Best for LLMs

Open Webpage

Shane Bergsma, Nolan Simran Dey, Gurpreet Gosal, Gavia Gray, Daria Soboleva, Joel Hestness

Published: 01 Jan 2025, Last Modified: 08 Oct 2025ICLR 2025EveryoneRevisionsBibTeXCC BY-SA 4.0