Aitor Lewkowycz

Postdoc, Google

Joined

September 2020

Names

Aitor Lewkowycz (Preferred)

Emails

****@google.com (Confirmed)

,

****@gmail.com (Confirmed)

Personal Links

Career & Education History

Postdoc

Google (google.com)

2019 – Present

Advisors, Relations & Conflicts

No relations added

Expertise

Understanding deep learning

2019 – Present

String Theory

2011 – 2018

Publications

Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R. Brown, Adam Santoro, Aditya Gupta, Adrià Garriga-Alonso, Agnieszka Kluska, Aitor Lewkowycz, Akshat Agarwal, Alethea Power, Alex Ray, Alex Warstadt, Alexander W. Kocurek, Ali Safaya, Ali Tazarv, Alice Xiang et al. (430 additional authors not shown)
- Accepted by TMLR
- Readers: Everyone
Exploring Length Generalization in Large Language Models
Cem Anil, Yuhuai Wu, Anders Johan Andreassen, Aitor Lewkowycz, Vedant Misra, Vinay Venkatesh Ramasesh, Ambrose Slone, Guy Gur-Ari, Ethan Dyer, Behnam Neyshabur
- Published: 31 Oct 2022, Last Modified: 16 Dec 2022
- NeurIPS 2022 Accept
- Readers: Everyone
Solving Quantitative Reasoning Problems with Language Models
Aitor Lewkowycz, Anders Johan Andreassen, David Dohan, Ethan Dyer, Henryk Michalewski, Vinay Venkatesh Ramasesh, Ambrose Slone, Cem Anil, Imanol Schlag, Theo Gutman-Solo, Yuhuai Wu, Behnam Neyshabur, Guy Gur-Ari, Vedant Misra
- Published: 31 Oct 2022, Last Modified: 14 Oct 2022
- NeurIPS 2022 Accept
- Readers: Everyone
Show Your Work: Scratchpads for Intermediate Computation with Language Models
Maxwell Nye, Anders Johan Andreassen, Guy Gur-Ari, Henryk Michalewski, Jacob Austin, David Bieber, David Dohan, Aitor Lewkowycz, Maarten Bosma, David Luan, Charles Sutton, Augustus Odena
- Published: 26 Mar 2022, Last Modified: 05 May 2023
- DL4C 2022
- Readers: Everyone
Show Your Work: Scratchpads for Intermediate Computation with Language Models
Maxwell Nye, Anders Johan Andreassen, Guy Gur-Ari, Henryk Michalewski, Jacob Austin, David Bieber, David Dohan, Aitor Lewkowycz, Maarten Bosma, David Luan, Charles Sutton, Augustus Odena
- Published: 28 Jan 2022, Last Modified: 13 Feb 2023
- ICLR 2022 Submitted
- Readers: Everyone
How to decay your learning rate
Aitor Lewkowycz
- Published: 28 Jan 2022, Last Modified: 12 Oct 2025
- ICLR 2022 Submitted
- Readers: Everyone
Effect of scale on catastrophic forgetting in neural networks
Vinay Venkatesh Ramasesh, Aitor Lewkowycz, Ethan Dyer
- Published: 28 Jan 2022, Last Modified: 13 Feb 2023
- ICLR 2022 Poster
- Readers: Everyone
The large learning rate phase of deep learning
Aitor Lewkowycz, Yasaman Bahri, Ethan Dyer, Jascha Sohl-Dickstein, Guy Gur-Ari
- 28 Sept 2020 (modified: 12 Oct 2025)
- ICLR 2021 Conference Blind Submission
- Readers: Everyone
The large learning rate phase of deep learning: the catapult mechanism
Aitor Lewkowycz, Yasaman Bahri, Ethan Dyer, Jascha Sohl-Dickstein, Guy Gur-Ari
- 2020 (modified: 06 Nov 2022)
- CoRR 2020
- Readers: Everyone
On the training dynamics of deep networks with L2 regularization
Aitor Lewkowycz, Guy Gur-Ari
- 2020 (modified: 06 Nov 2022)
- CoRR 2020
- Readers: Everyone

Co-Authors

View all 460 co-authors