William Fedus

Joined

October 2017

Names

William Fedus (Preferred)

Emails

****@gmail.com (Confirmed)

****@google.com (Confirmed)

Personal Links

Career & Education History

University of Montreal (umontrea)

2017 – 2020

MS student

University of California, San Diego (ucsd)

2013 – 2016

Undergrad student

Massachusetts Institute of Technology (mit)

2006 – 2010

Advisors, Relations & Conflicts

PhD Advisor

Yoshua Bengio

2017 – 2020

PhD Advisor

Hugo Larochelle

2017 – 2020

Expertise

Deep learning

Present

Publications

Mixture-of-experts meets instruction tuning: A winning combination for large language models
Sheng Shen, Le Hou, Yanqi Zhou, Nan Du, Shayne Longpre, Jason Wei, Hyung Won Chung, Barret Zoph, William Fedus, Xinyun Chen, Tu Vu, Yuexin Wu, Wuyang Chen, Albert Webson, Yunxuan Li, Vincent Y Zhao, Hongkun Yu, Kurt Keutzer, Trevor Darrell, Denny Zhou
- 16 Feb 2024
- OpenReview Archive Direct Upload
- Readers: Everyone
Mixture-of-Experts Meets Instruction Tuning: A Winning Combination for Large Language Models
Sheng Shen, Le Hou, Yanqi Zhou, Nan Du, Shayne Longpre, Jason Wei, Hyung Won Chung, Barret Zoph, William Fedus, Xinyun Chen, Tu Vu, Yuexin Wu, Wuyang Chen, Albert Webson, Yunxuan Li, Vincent Y Zhao, Hongkun Yu, Kurt Keutzer, Trevor Darrell, Denny Zhou
- ICLR 2024 poster
- Readers: Everyone
Scaling Laws vs Model Architectures: How does Inductive Bias Influence Scaling?
Yi Tay, Mostafa Dehghani, Samira Abnar, Hyung Won Chung, William Fedus, Jinfeng Rao, Sharan Narang, Vinh Q. Tran, Dani Yogatama, Donald Metzler
- EMNLP 2023 Findings
- Readers: Everyone
Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R. Brown, Adam Santoro, Aditya Gupta, Adrià Garriga-Alonso, Agnieszka Kluska, Aitor Lewkowycz, Akshat Agarwal, Alethea Power, Alex Ray, Alex Warstadt, Alexander W. Kocurek, Ali Safaya, Ali Tazarv, Alice Xiang et al. (430 additional authors not shown)
- Accepted by TMLR
- Readers: Everyone
Scaling Laws vs Model Architectures: How does Inductive Bias Influence Scaling?
Yi Tay, Mostafa Dehghani, Samira Abnar, Hyung Won Chung, William Fedus, Jinfeng Rao, Sharan Narang, Vinh Q. Tran, Dani Yogatama, Donald Metzler
- Published: 01 Feb 2023, Last Modified: 13 Feb 2023
- Submitted to ICLR 2023
- Readers: Everyone
Emergent Abilities of Large Language Models
Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, Ed H. Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, William Fedus
- Accepted by TMLR
- Readers: Everyone
Scale Efficiently: Insights from Pretraining and Finetuning Transformers
Yi Tay, Mostafa Dehghani, Jinfeng Rao, William Fedus, Samira Abnar, Hyung Won Chung, Sharan Narang, Dani Yogatama, Ashish Vaswani, Donald Metzler
- Published: 28 Jan 2022, Last Modified: 13 Feb 2023
- ICLR 2022 Poster
- Readers: Everyone
Revisiting ResNets: Improved Training and Scaling Strategies
Irwan Bello, William Fedus, Xianzhi Du, Ekin Dogus Cubuk, Aravind Srinivas, Tsung-Yi Lin, Jonathon Shlens, Barret Zoph
- Published: 09 Nov 2021, Last Modified: 26 May 2025
- NeurIPS 2021 Spotlight
- Readers: Everyone
Revisiting fundamentals of experience replay
William Fedus, Prajit Ramachandran, Rishabh Agarwal, Yoshua Bengio, Hugo Larochelle, Mark Rowland, Will Dabney
- 11 May 2021 (modified: 05 Jul 2022)
- OpenReview Archive Direct Upload
- Readers: Everyone
Disentangling the independently controllable factors of variation by interacting with the world
Valentin Thomas, Emmanuel Bengio, William Fedus, Jules Pondard, Philippe Beaudoin, Hugo Larochelle, Joelle Pineau, Doina Precup, Yoshua Bengio
- 10 May 2021
- OpenReview Archive Direct Upload
- Readers: Everyone

Co-Authors

View all 497 co-authors