Guilherme Penedo

Researcher, HuggingFace

Joined

March 2023

Names

Guilherme Penedo (Preferred)

Emails

****@lighton.ai (Confirmed)

****@gmail.com (Confirmed)

****@huggingface.co (Confirmed)

****@macrodata.co (Confirmed)

Personal Links

Homepage

Google Scholar

Semantic Scholar

Career & Education History

Researcher

HuggingFace (huggingface.co)

2023 – 2026

Researcher

LightOn (lighton.ai)

2022 – 2023

MS student

Institut Supérieur de l'Aéronautique et de l'Espace (isae-supaero.fr)

2020 – 2022

Advisors, Relations & Conflicts

No relations added

Expertise

No areas of expertise listed

Publications

The Common Pile v0.1: An 8TB Dataset of Public Domain and Openly Licensed Text
Nikhil Kandpal, Brian Lester, Colin Raffel, Sebastian Majstorovic, Stella Biderman, Baber Abbasi, Luca Soldaini, Enrico Shippole, A. Feder Cooper, Aviya Skowron, Shayne Longpre, Lintang Sutawika, Alon Albalak, Zhenlin Xu, Guilherme Penedo, Loubna Ben allal, Elie Bakouch, John David Pressman, Honglu Fan, Dashiell Stander et al. (7 additional authors not shown)
- NeurIPS 2025 Datasets and Benchmarks Track poster
- Readers: Everyone
FineWeb2: One Pipeline to Scale Them All — Adapting Pre-Training Data Processing to Every Language
Guilherme Penedo, Hynek Kydlíček, Vinko Sabolčec, Bettina Messmer, Negar Foroutan, Amir Hossein Kargaran, Colin Raffel, Martin Jaggi, Leandro Von Werra, Thomas Wolf
- COLM 2025
- Readers: Everyone
SmolLM2: When Smol Goes Big — Data-Centric Training of a Fully Open Small Language Model
Loubna Ben allal, Anton Lozhkov, Elie Bakouch, Gabriel Martin Blazquez, Guilherme Penedo, Lewis Tunstall, Andrés Marafioti, Agustín Piqueres Lajarín, Hynek Kydlíček, Vaibhav Srivastav, Joshua Lochner, Caleb Fahlgren, Xuan Son NGUYEN, Ben Burtenshaw, Clémentine Fourrier, Haojun Zhao, Hugo Larcher, Mathieu Morlon, Cyril Zakka, Colin Raffel et al. (2 additional authors not shown)
- COLM 2025
- Readers: Everyone
The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale
Guilherme Penedo, Hynek Kydlíček, Loubna Ben allal, Anton Lozhkov, Margaret Mitchell, Colin Raffel, Leandro Von Werra, Thomas Wolf
- NeurIPS 2024 Track Datasets and Benchmarks Spotlight
- Readers: Everyone
The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data Only
Guilherme Penedo, Quentin Malartic, Daniel Hesslow, Ruxandra Cojocaru, Hamza Alobeidli, Alessandro Cappelli, Baptiste Pannier, Ebtesam Almazrouei, Julien Launay
- NeurIPS 2023 Datasets and Benchmarks Poster
- Readers: Everyone

Co-Authors

View all 58 co-authors

Guilherme Penedo

Names

Emails

Personal Links

Career & Education History

Advisors, Relations & Conflicts

Expertise

Publications

The Common Pile v0.1: An 8TB Dataset of Public Domain and Openly Licensed Text

FineWeb2: One Pipeline to Scale Them All — Adapting Pre-Training Data Processing to Every Language

SmolLM2: When Smol Goes Big — Data-Centric Training of a Fully Open Small Language Model

The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale

The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data Only

Co-Authors