Jeffrey Pennington

Joined

November 2016

Names

Jeffrey Pennington (Preferred)

Emails

****@google.com (Confirmed)

****@gmail.com (Confirmed)

Personal Links

Google Scholar

DBLP

Career & Education History

Research Scientist

Google (google.com)

2014 – 2025

Postdoc

Stanford University (stanford.edu)

2013 – 2014

PhD student

Stanford University (stanford.edu)

2007 – 2013

Undergrad student

University of Southern California (usc.edu)

2003 – 2007

Advisors, Relations & Conflicts

No relations added

Expertise

Theory of deep learning

2013 – 2020

Geometry of neural networks

2016 – 2016

Recurrent neural networks

2015 – 2016

Word embeddings

2013 – 2016

Theoretical understanding of neural networks

2013 – 2016

Recursive Neural Networks

2011 – 2011

Sentiment analysis

2011 – 2011

Publications

Scaling Collapse Reveals Universal Dynamics in Compute-Optimally Trained Neural Networks
Shikai Qiu, Lechao Xiao, Andrew Gordon Wilson, Jeffrey Pennington, Atish Agarwala
- CoRR 2025
- Readers: Everyone
Scaling Collapse Reveals Universal Dynamics in Compute-Optimally Trained Neural Networks
Shikai Qiu, Lechao Xiao, Andrew Gordon Wilson, Jeffrey Pennington, Atish Agarwala
- ICML 2025 oral
- Readers: Everyone
Scaling Collapse Reveals Universal Dynamics in Compute-Optimally Trained Neural Networks
Shikai Qiu, Atish Agarwala, Jeffrey Pennington, Lechao Xiao
- NeurIPS 2024 Workshop
- Readers: Everyone
Training LLMs over Neurally Compressed Text
Brian Lester, Jaehoon Lee, Alexander A Alemi, Jeffrey Pennington, Adam Roberts, Jascha Sohl-Dickstein, Noah Constant
- Accepted by TMLR
- Readers: Everyone
4+3 Phases of Compute-Optimal Neural Scaling Laws
Elliot Paquette, Courtney Paquette, Lechao Xiao, Jeffrey Pennington
- NeurIPS 2024 spotlight
- Readers: Everyone
Training Language Models on the Knowledge Graph: Insights on Hallucinations and Their Detectability
Jiri Hron, Laura A Culp, Gamaleldin Fathy Elsayed, Rosanne Liu, Jasper Snoek, Simon Kornblith, Alex Rizkowsky, Isabelle Simpson, Jascha Sohl-Dickstein, Noah Fiedel, Aaron T Parisi, Alexander A Alemi, Azade Nova, Ben Adlam, Bernd Bohnet, Gaurav Mishra, Hanie Sedghi, Izzeddin Gur, Jaehoon Lee, John D Co-Reyes et al. (11 additional authors not shown)
- COLM
- Readers: Everyone
Scaling Exponents Across Parameterizations and Optimizers
Katie E Everett, Lechao Xiao, Mitchell Wortsman, Alexander A Alemi, Roman Novak, Peter J Liu, Izzeddin Gur, Jascha Sohl-Dickstein, Leslie Pack Kaelbling, Jaehoon Lee, Jeffrey Pennington
- ICML 2024 Poster
- Readers: Everyone
Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Models
Avi Singh, John D Co-Reyes, Rishabh Agarwal, Ankesh Anand, Piyush Patil, Xavier Garcia, Peter J Liu, James Harrison, Jaehoon Lee, Kelvin Xu, Aaron T Parisi, Abhishek Kumar, Alexander A Alemi, Alex Rizkowsky, Azade Nova, Ben Adlam, Bernd Bohnet, Gamaleldin Fathy Elsayed, Hanie Sedghi, Igor Mordatch et al. (20 additional authors not shown)
- Accepted by TMLR
- Readers: Everyone
Small-scale proxies for large-scale Transformer training instabilities
Mitchell Wortsman, Peter J. Liu, Lechao Xiao, Katie E. Everett, Alexander A. Alemi, Ben Adlam, John D. Co-Reyes, Izzeddin Gur, Abhishek Kumar, Roman Novak, Jeffrey Pennington, Jascha Sohl-Dickstein, Kelvin Xu, Jaehoon Lee, Justin Gilmer, Simon Kornblith
- ICLR 2024
- Readers: Everyone
Scaling Exponents Across Parameterizations and Optimizers
Katie Everett, Lechao Xiao, Mitchell Wortsman, Alexander A. Alemi, Roman Novak, Peter J. Liu, Izzeddin Gur, Jascha Sohl-Dickstein, Leslie Pack Kaelbling, Jaehoon Lee, Jeffrey Pennington
- CoRR 2024
- Readers: Everyone

View all 93 publications

Co-Authors

View all 110 co-authors