Multiple Descents in Deep Learning as a Sequence of Order-Chaos Transitions

WEIWENBO, Nicholas Jia Le Chong, CHOY HENG LAI, Ling Feng

Keywords: Learning dynamics, double descent, multiple descent, critical transitions

TL;DR: This paper introduces a "multiple descent" phenomenon in LSTM networks, where test loss cycles repeatedly during training due to order-chaos phase transitions, with optimal performance occurring at the first broad transition from order to chaos.

Confirmation Of Submission Requirements: I submit an abstract. It uses the template provided on the submission page and is no longer than 2 pages.

Submission Number: 103