Alon Albalak

Researcher, Lila Sciences

Joined

November 2021

Names

Alon Albalak (Preferred)

Emails

****@ucsb.edu (Confirmed)

****@gmail.com (Confirmed)

Personal Links

Career & Education History

Researcher

Lila Sciences (lila.ai)

2025 – Present

Researcher

SynthLabs (synthlabs.ai)

2024 – 2025

PhD student

University of California, Santa Barbara (ucsb.edu)

2018 – 2024

Advisors, Relations & Conflicts

PhD Advisor

William Yang Wang

Present

PhD Advisor

Xifeng Yan

Present

Expertise

creativity

2025 – Present

synthetic data

post-training

reasoning

reinforcement learning

2024 – Present

language modelling

data selection

data deduplication

data mixing

2023 – Present

transfer learning

domain adaptation

task transfer

multitask learning

2021 – Present

data efficiency

2021 – Present

dialogue systems

information extraction

2020 – 2022

Publications

MAGPIE: A benchmark for Multi-AGent contextual PrIvacy Evaluation
Gurusha Juneja, Jayanth Pasupulati, Alon Albalak, Wenyue Hua, William Yang Wang
- ResponsibleFM @ NeurIPS 2025
- Readers: Everyone
OpenThoughts: Data Recipes for Reasoning Models
Etash Kumar Guha, Ryan Marten, Sedrick Keh, Negin Raoof, Georgios Smyrnis, Hritik Bansal, Marianna Nezhurina, Jean Mercat, Trung Vu, Zayne Rea Sprague, Ashima Suvarna, Benjamin Feuer, Leon Liangyu Chen, Zaid Khan, Eric Frankel, Sachin Grover, Caroline Choi, Niklas Muennighoff, Shiye Su, Wanjia Zhao et al. (31 additional authors not shown)
- ICLR 2026 Oral
- Readers: Everyone
OpenThoughts: Data Recipes for Reasoning Models
Etash Kumar Guha, Ryan Marten, Sedrick Keh, Negin Raoof, Georgios Smyrnis, Hritik Bansal, Marianna Nezhurina, Jean Mercat, Trung Vu, Zayne Rea Sprague, Ashima Suvarna, Benjamin Feuer, Leon Liangyu Chen, Zaid Khan, Eric Frankel, Sachin Grover, Caroline Choi, Niklas Muennighoff, Shiye Su, Wanjia Zhao et al. (30 additional authors not shown)
- FoRLM 2025
- Readers: Everyone
The Common Pile v0.1: An 8TB Dataset of Public Domain and Openly Licensed Text
Nikhil Kandpal, Brian Lester, Colin Raffel, Sebastian Majstorovic, Stella Biderman, Baber Abbasi, Luca Soldaini, Enrico Shippole, A. Feder Cooper, Aviya Skowron, Shayne Longpre, Lintang Sutawika, Alon Albalak, Zhenlin Xu, Guilherme Penedo, Loubna Ben allal, Elie Bakouch, John David Pressman, Honglu Fan, Dashiell Stander et al. (7 additional authors not shown)
- NeurIPS 2025 Datasets and Benchmarks Track poster
- Readers: Everyone
Generative Reward Models
Dakota Mahan, Duy Van Phung, Alon Albalak, Rafael Rafailov, Chase Blagden, Nathan Lile, Louis Castricato, Jan-Philipp Fränken, Chelsea Finn
- Submitted to ICLR 2025
- Readers: Everyone
Generalization v.s. Memorization: Tracing Language Models’ Capabilities Back to Pretraining Data
Xinyi Wang, Antonis Antoniades, Yanai Elazar, Alfonso Amayuelas, Alon Albalak, Kexun Zhang, William Yang Wang
- ICLR 2025 Poster
- Readers: Everyone
Eagle and Finch: RWKV with Matrix-Valued States and Dynamic Recurrence
Bo Peng, Daniel Goldstein, Quentin Gregory Anthony, Alon Albalak, Eric Alcaide, Stella Biderman, Eugene Cheah, Xingjian Du, Teddy Ferdinan, Haowen Hou, Przemyslaw Kazienko, Kranthi Kiran GV, Jan Kocon, Bartłomiej Koptyra, Satyapriya Krishna, Ronald McClelland Jr., Niklas Muennighoff, Fares Obeid, Atsushi Saito, Guangyu Song et al. (8 additional authors not shown)
- OpenReview Archive Direct Upload
- Readers: Everyone
Rwkv: Reinventing rnns for the transformer era
Bo Peng, Eric Alcaide, Quentin Gregory Anthony, Alon Albalak, Samuel Arcadinho, Stella Biderman, Huanqi Cao, Xin Cheng, Michael Chungyoun, Matteo Grella, Kranthi Kiran GV, Xuzheng He, Haowen Hou, Jiaju Lin, Przemyslaw Kazienko, Jan Kocon, Jiaming Kong, Bartłomiej Koptyra, Hayden Lau, Krishna Sri Ipsit Mantri et al. (14 additional authors not shown)
- OpenReview Archive Direct Upload
- Readers: Everyone
The Responsible Foundation Model Development Cheatsheet: A Review of Tools & Resources
Shayne Longpre, Stella Biderman, Alon Albalak, Hailey Schoelkopf, Daniel McDuff, Sayash Kapoor, Kevin Klyman, Kyle Lo, Gabriel Ilharco, Nay San, Maribeth Rauh, Aviya Skowron, Bertie Vidgen, Laura Weidinger, Arvind Narayanan, Victor Sanh, David Ifeoluwa Adelani, Percy Liang, Rishi Bommasani, Peter Henderson et al. (3 additional authors not shown)
- Accepted by TMLR
- Readers: Everyone
Generalization vs. Memorization: Tracing Language Models' Capabilities Back to Pretraining Data
Antonis Antoniades, Xinyi Wang, Yanai Elazar, Alfonso Amayuelas, Alon Albalak, Kexun Zhang, William Yang Wang
- ICML 2024 FM-Wild Workshop Poster
- Readers: Everyone

View all 36 publications

Co-Authors

View all 220 co-authors