Shane Bergsma

Principal Researcher, Cerebras Systems, Inc

Joined

July 2019

Names

Shane Bergsma

Emails

****@gmail.com (Confirmed)

****@cerebras.net (Confirmed)

Personal Links

Homepage

Google Scholar

DBLP

Career & Education History

Principal Researcher

Cerebras Systems, Inc (cerebras.net)

2024 – Present

Researcher

Huawei Canada (huawei.com)

2014 – 2024

Advisors, Relations & Conflicts

PhD Advisor

Dekang Lin

Present

Postdoc Advisor

Kenneth Church

Present

Expertise

No areas of expertise listed

Publications

Scaling with Collapse: Efficient and Predictable Training of LLM Families
Shane Bergsma, Bin Claire Zhang, Nolan Simran Dey, Shaheer Muhammad, Gurpreet Gosal, Joel Hestness
- ICLR 2026 Poster
- Readers: Everyone
Predicting Training Re-evaluation Curves Enables Effective Data Curriculums for LLMs
Shane Bergsma, Nolan Simran Dey, Joel Hestness
- ICLR 2026 Poster
- Readers: Everyone
PTPP-Aware Adaptation Scaling Laws: Predicting Domain-Adaptation Performance at Unseen Pre-Training Budgets
Etienne Goffinet, Shane Bergsma, Avraham Sheinin, Natalia Vassilieva, Preslav Nakov, Gurpreet Gosal
- CCFM Poster
- Readers: Everyone
Power Lines: Scaling laws for weight decay and batch size in LLM pre-training
Shane Bergsma, Nolan Simran Dey, Gurpreet Gosal, Gavia Gray, Daria Soboleva, Joel Hestness
- NeurIPS 2025 poster
- Readers: Everyone
Don't be lazy: CompleteP enables compute-efficient deep transformers
Nolan Simran Dey, Bin Claire Zhang, Lorenzo Noci, Mufan Li, Blake Bordelon, Shane Bergsma, Cengiz Pehlevan, Boris Hanin, Joel Hestness
- NeurIPS 2025 poster
- Readers: Everyone
Sparse maximal update parameterization: A holistic approach to sparse training dynamics
Nolan Simran Dey, Shane Bergsma, Joel Hestness
- NeurIPS 2024
- Readers: Everyone
Empirical Upper Bounds for Unstructured Sparsity in Compute-Efficient Language Modeling
Esha Singh, Shane Bergsma, Nolan Simran Dey, Joel Hestness, Gavia Gray
- Compression Workshop @ NeurIPS 2024
- Readers: Everyone
Straight to Zero: Why Linearly Decaying the Learning Rate to Zero Works Best for LLMs
Shane Bergsma, Nolan Simran Dey, Gurpreet Gosal, Gavia Gray, Daria Soboleva, Joel Hestness
- ICLR 2025 Poster
- Readers: Everyone
Normalization Layer Per-Example Gradients are Sufficient to Predict Gradient Noise Scale in Transformers
Gavia Gray, Aman Tiwari, Shane Bergsma, Joel Hestness
- NeurIPS 2024 poster
- Readers: Everyone
Sparse maximal update parameterization: A holistic approach to sparse training dynamics
Nolan Simran Dey, Shane Bergsma, Joel Hestness
- NeurIPS 2024 poster
- Readers: Everyone

View all 30 publications

Co-Authors

View all 42 co-authors