Samuel Weinbach

Researcher, Aleph Alpha GmbH

Joined

March 2022

Names

Samuel Weinbach (Preferred)

Emails

****@gmail.com (Confirmed)

Personal Links

Career & Education History

Researcher

Aleph Alpha GmbH (aleph-alpha.com)

2019 – Present

Advisors, Relations & Conflicts

No relations added

Expertise

No areas of expertise listed

Publications

u-$\mu$P: The Unit-Scaled Maximal Update Parametrization
Charlie Blake, Constantin Eichenberg, Josef Dean, Lukas Balles, Luke Yuri Prince, Björn Deiseroth, Andres Felipe Cruz-Salinas, Carlo Luschi, Samuel Weinbach, Douglas Orr
- NeurIPS 2024 Workshop
- Readers: Everyone
u-$\mu$P: The Unit-Scaled Maximal Update Parametrization
Charlie Blake, Constantin Eichenberg, Josef Dean, Lukas Balles, Luke Yuri Prince, Björn Deiseroth, Andres Felipe Cruz-Salinas, Carlo Luschi, Samuel Weinbach, Douglas Orr
- ICLR 2025 Spotlight
- Readers: Everyone
u-μP: The Unit-Scaled Maximal Update Parametrization
Charlie Blake, Constantin Eichenberg, Josef Dean, Lukas Balles, Luke Yuri Prince, Björn Deiseroth, Andres Felipe Cruz-Salinas, Carlo Luschi, Samuel Weinbach, Douglas Orr
- WANT@ICML 2024 Poster
- Readers: Everyone
u-μP: The Unit-Scaled Maximal Update Parametrization
Charlie Blake, Constantin Eichenberg, Josef Dean, Lukas Balles, Luke Yuri Prince, Björn Deiseroth, Andres Felipe Cruz-Salinas, Carlo Luschi, Samuel Weinbach, Douglas Orr
- HiLD at ICML 2024 Poster
- Readers: Everyone
Efficient Parallelization Layouts for Large-Scale Distributed Model Training
Johannes Hagemann, Samuel Weinbach, Konstantin Dobler, Maximilian Schall, Gerard de Melo
- COLM
- Readers: Everyone
Efficient Parallelization Layouts for Large-Scale Distributed Model Training
Johannes Hagemann, Samuel Weinbach, Konstantin Dobler, Maximilian Schall, Gerard de Melo
- WANT@NeurIPS 2023 Oral
- Readers: Everyone
MultiFusion: Fusing Pre-Trained Models for Multi-Lingual, Multi-Modal Image Generation
Marco Bellagente, Manuel Brack, Hannah Benita Teufel, Felix Friedrich, Björn Deiseroth, Constantin Eichenberg, Andrew Dai, Robert John Nicholas Baldock, Souradeep Nanda, Koen Oostermeijer, Andres Felipe Cruz-Salinas, Patrick Schramowski, Kristian Kersting, Samuel Weinbach
- NeurIPS 2023 poster
- Readers: Everyone
ATMAN: Understanding Transformer Predictions Through Memory Efficient Attention Manipulation
Björn Deiseroth, Mayukh Deb, Samuel Weinbach, Manuel Brack, Patrick Schramowski, Kristian Kersting
- NeurIPS 2023 poster
- Readers: Everyone
AtMan: Understanding Transformer Predictions Through Memory Efficient Attention Manipulation
Mayukh Deb, Björn Deiseroth, Samuel Weinbach, Patrick Schramowski, Kristian Kersting
- 2023 (modified: 18 Apr 2023)
- CoRR 2023
- Readers: Everyone
GPT-NeoX-20B: An Open-Source Autoregressive Language Model
Sidney Black, Stella Biderman, Eric Hallahan, Quentin Gregory Anthony, Leo Gao, Laurence Golding, Horace He, Connor Leahy, Kyle McDonell, Jason Phang, Michael Martin Pieler, USVSN Sai Prashanth, Shivanshu Purohit, Laria Reynolds, Jonathan Tow, Ben Wang, Samuel Weinbach
- Published: 09 Apr 2022, Last Modified: 08 Feb 2026
- BigScience#5
- Readers: Everyone

View all 15 publications

Co-Authors

View all 52 co-authors