Zachary Nado

Research Engineer, Google

Joined

February 2018

Names

Zachary Nado

Emails

****@google.com (Confirmed)

****@alumni.brown.edu (Confirmed)

Personal Links

Homepage

Google Scholar

DBLP

Career & Education History

Research Engineer

Google (google.com)

2016 – Present

Advisors, Relations & Conflicts

No relations added

Expertise

Preconditioned Optimization

Present

Hyperparameter Tuning

Present

Neural Network Optimization

Present

Publications

A Little Help Goes a Long Way: Efficient LLM Training by Leveraging Small LMs
Ankit Singh Rawat, Veeranjaneyulu Sadhanala, Afshin Rostamizadeh, Ayan Chakrabarti, Wittawat Jitkrittum, Vladimir Feinberg, Seungyeon Kim, Hrayr Harutyunyan, Nikunj Saunshi, Zachary Nado, Rakesh Shivanna, Sashank J. Reddi, Aditya Krishna Menon, Rohan Anil, Sanjiv Kumar
- Submitted to ICLR 2026
- Readers: Everyone
Pre-trained Gaussian Processes for Bayesian Optimization
Zi Wang, George E. Dahl, Kevin Swersky, Chansoo Lee, Zachary Nado, Justin Gilmer, Jasper Snoek, Zoubin Ghahramani
- Journal of Machine Learning Research
- Readers: Everyone
A Little Help Goes a Long Way: Efficient LLM Training by Leveraging Small LMs
Ankit Singh Rawat, Veeranjaneyulu Sadhanala, Afshin Rostamizadeh, Ayan Chakrabarti, Wittawat Jitkrittum, Vladimir Feinberg, Seungyeon Kim, Hrayr Harutyunyan, Nikunj Saunshi, Zachary Nado, Rakesh Shivanna, Sashank J. Reddi, Aditya Krishna Menon, Rohan Anil, Sanjiv Kumar
- Submitted to ICLR 2025
- Readers: Everyone
Accelerating neural network training: An analysis of the AlgoPerf competition
Priya Kasimbeg, Frank Schneider, Runa Eschenhagen, Juhan Bae, Chandramouli Shama Sastry, Mark Saroufim, BOYUAN FENG, Less Wright, Edward Z. Yang, Zachary Nado, Sourabh Medapati, Philipp Hennig, Michael Rabbat, George E. Dahl
- ICLR 2025 Poster
- Readers: Everyone
Adaptive Gradient Methods at the Edge of Stability
Jeremy Cohen, Behrooz Ghorbani, Shankar Krishnan, Naman Agarwal, Sourabh Medapati, Michal Badura, Daniel Suo, Zachary Nado, George E. Dahl, Justin Gilmer
- HeavyTails 2023
- Readers: Everyone
Kernel Regression with Infinite-Width Neural Networks on Millions of Examples
Ben Adlam, Jaehoon Lee, Shreyas Padhy, Zachary Nado, Jasper Snoek
- Published: 01 Feb 2023, Last Modified: 14 Jan 2026
- Submitted to ICLR 2023
- Readers: Everyone
Plex: Towards Reliability using Pretrained Large Model Extensions
Dustin Tran, Jeremiah Zhe Liu, Michael W Dusenberry, Du Phan, Mark Collier, Jie Ren, Kehang Han, Zi Wang, Zelda E Mariet, Huiyi Hu, Neil Band, Tim G. J. Rudner, Karan Singhal, Zachary Nado, Joost van Amersfoort, Andreas Kirsch, Rodolphe Jenatton, Nithum Thain, Honglin Yuan, E. Kelly Buchanan et al. (6 additional authors not shown)
- 26 May 2022 (modified: 30 Dec 2025)
- ICML 2022 Pre-training Workshop ContributedTalk
- Readers: Everyone
Benchmarking Bayesian Deep Learning on Diabetic Retinopathy Detection Tasks
Neil Band, Tim G. J. Rudner, Qixuan Feng, Angelos Filos, Zachary Nado, Michael W Dusenberry, Ghassen Jerfel, Dustin Tran, Yarin Gal
- Published: 02 Dec 2021, Last Modified: 04 May 2025
- NeurIPS 2021 Workshop DistShift Spotlight
- Readers: Everyone
A Large Batch Optimizer Reality Check: Traditional, Generic Optimizers Suffice Across Batch Sizes
Zachary Nado, Justin Gilmer, Christopher J Shallue, Rohan Anil, George Edward Dahl
- Published: 28 Jan 2022, Last Modified: 12 Oct 2025
- ICLR 2022 Submitted
- Readers: Everyone
A Loss Curvature Perspective on Training Instabilities of Deep Learning Models
Justin Gilmer, Behrooz Ghorbani, Ankush Garg, Sneha Kudugunta, Behnam Neyshabur, David Cardoze, George Edward Dahl, Zachary Nado, Orhan Firat
- Published: 28 Jan 2022, Last Modified: 12 Oct 2025
- ICLR 2022 Poster
- Readers: Everyone

View all 17 publications

Co-Authors

View all 81 co-authors