Soham De

Research Scientist, DeepMind

Joined

March 2019

Names

Soham De (Preferred)

Emails

****@cs.umd.edu (Confirmed)

****@google.com (Confirmed)

****@gmail.com (Confirmed)

Personal Links

Homepage

Google Scholar

DBLP

Career & Education History

Research Scientist

DeepMind (google.com)

2018 – Present

PhD student

Department of Computer Science, University of Maryland, College Park (cs.umd.edu)

2013 – 2018

Advisors, Relations & Conflicts

No relations added

Expertise

No areas of expertise listed

Publications

How do language models learn facts? Dynamics, curricula and hallucinations
Nicolas Zucchet, Jorg Bornschein, Stephanie C.Y. Chan, Razvan Pascanu, Andrew Kyle Lampinen, Soham De
- OpenReview Archive Direct Upload
- Readers: Everyone
How do language models learn facts? Dynamics, curricula and hallucinations
Nicolas Zucchet, Jorg Bornschein, Stephanie C.Y. Chan, Andrew Kyle Lampinen, Razvan Pascanu, Soham De
- COLM 2025
- Readers: Everyone
Universality of Linear Recurrences Followed by Non-linear Projections: Finite-Width Guarantees and Benefits of Complex Eigenvalues
Antonio Orvieto, Soham De, Caglar Gulcehre, Razvan Pascanu, Samuel L Smith
- ICML 2024 Poster
- Readers: Everyone
Universality of Linear Recurrences Followed by Non-linear Projections: Finite-Width Guarantees and Benefits of Complex Eigenvalues
Antonio Orvieto, Soham De, Caglar Gulcehre, Razvan Pascanu, Samuel L. Smith
- ICML 2024
- Readers: Everyone
Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models
Soham De, Samuel L. Smith, Anushan Fernando, Aleksandar Botev, George-Cristian Muraru, Albert Gu, Ruba Haroun, Leonard Berrada, Yutian Chen, Srivatsan Srinivasan, Guillaume Desjardins, Arnaud Doucet, David Budden, Yee Whye Teh, Razvan Pascanu, Nando de Freitas, Caglar Gulcehre
- CoRR 2024
- Readers: Everyone
RecurrentGemma: Moving Past Transformers for Efficient Open Language Models
Aleksandar Botev, Soham De, Samuel L. Smith, Anushan Fernando, George-Cristian Muraru, Ruba Haroun, Leonard Berrada, Razvan Pascanu, Pier Giuseppe Sessa, Robert Dadashi, Léonard Hussenot, Johan Ferret, Sertan Girgin, Olivier Bachem, Alek Andreev, Kathleen Kenealy, Thomas Mesnard, Cassidy Hardin, Surya Bhupatiraju, Shreya Pathak et al. (42 additional authors not shown)
- CoRR 2024
- Readers: Everyone
Resurrecting Recurrent Neural Networks for Long Sequences
Antonio Orvieto, Samuel L Smith, Albert Gu, Anushan Fernando, Caglar Gulcehre, Razvan Pascanu, Soham De
- ICML 2023 OralPoster
- Readers: Everyone
Unlocking Accuracy and Fairness in Differentially Private Image Classification
Leonard Berrada, Soham De, Judy Hanwen Shen, Jamie Hayes, Robert Stanforth, David Stutz, Pushmeet Kohli, Samuel L. Smith, Borja Balle
- CoRR 2023
- Readers: Everyone
Resurrecting Recurrent Neural Networks for Long Sequences
Antonio Orvieto, Samuel L. Smith, Albert Gu, Anushan Fernando, Çaglar Gülçehre, Razvan Pascanu, Soham De
- ICML 2023
- Readers: Everyone
ConvNets Match Vision Transformers at Scale
Samuel L. Smith, Andrew Brock, Leonard Berrada, Soham De
- CoRR 2023
- Readers: Everyone

View all 38 publications

Co-Authors

View all 139 co-authors