Payal Bajaj

Joined

September 2019

Names

Payal Bajaj (Preferred)

Emails

****@microsoft.com (Confirmed)

Personal Links

Homepage

Career & Education History

Applied Scientist

Microsoft (microsoft.com)

2018 – 2019

MS student

Stanford University (stanford.edu)

2016 – 2018

Advisors, Relations & Conflicts

No relations added

Expertise

Deep Learning

Present

Conversational Search

Present

Large Scale Language Models

Present

Publications

Magneto: A Foundation Transformer
Hongyu Wang, Shuming Ma, Shaohan Huang, Li Dong, Wenhui Wang, Zhiliang Peng, Yu Wu, Payal Bajaj, Saksham Singhal, Alon Benhaim, Barun Patra, Zhun Liu, Vishrav Chaudhary, Xia Song, Furu Wei
- ICML 2023 Poster
- Readers: Everyone
Performance and Risk Trade-offs for Multi-word Text Prediction at Scale
Aniket Vashishtha, S. Sai Prasad, Payal Bajaj, Vishrav Chaudhary, Kate Cook, Sandipan Dandapat, Sunayana Sitaram, Monojit Choudhury
- Published: 01 Jan 2023, Last Modified: 09 May 2023
- EACL (Findings) 2023
- Readers: Everyone
Understanding the Effectiveness of Very Large Language Models on Dialog Evaluation
Jessica Huynh, Cathy Jiao, Prakhar Gupta, Shikib Mehri, Payal Bajaj, Vishrav Chaudhary, Maxine Eskénazi
- Published: 01 Jan 2023, Last Modified: 09 May 2023
- CoRR 2023
- Readers: Everyone
Pretraining One Language Model for All With the Text-To-Text Framework Using Model-Generated Signals
Linyuan Gong, Chenyan Xiong, Xiaodong Liu, Payal Bajaj, Yiqing Xie, Alvin Cheung, Jianfeng Gao, Xia Song
- 22 Sept 2022 (modified: 13 Feb 2023)
- ICLR 2023 Conference Withdrawn Submission
- Readers: Everyone
On the Representation Collapse of Sparse Mixture of Experts
Zewen Chi, Li Dong, Shaohan Huang, Damai Dai, Shuming Ma, Barun Patra, Saksham Singhal, Payal Bajaj, Xia Song, Xian-Ling Mao, Heyan Huang, Furu Wei
- Published: 31 Oct 2022, Last Modified: 11 Jan 2023
- NeurIPS 2022 Accept
- Readers: Everyone
Pretraining Text Encoders with Adversarial Mixture of Training Signal Generators
Yu Meng, Chenyan Xiong, Payal Bajaj, Saurabh Tiwary, Paul Bennett, Jiawei Han, Xia Song
- 2022 (modified: 22 Apr 2022)
- CoRR 2022
- Readers: Everyone
METRO: Efficient Denoising Pretraining of Large Scale Autoencoding Language Models with Model Generated Signals
Payal Bajaj, Chenyan Xiong, Guolin Ke, Xiaodong Liu, Di He, Saurabh Tiwary, Tie-Yan Liu, Paul Bennett, Xia Song, Jianfeng Gao
- 2022 (modified: 21 Sept 2022)
- CoRR 2022
- Readers: Everyone
Foundation Transformers
Hongyu Wang, Shuming Ma, Shaohan Huang, Li Dong, Wenhui Wang, Zhiliang Peng, Yu Wu, Payal Bajaj, Saksham Singhal, Alon Benhaim, Barun Patra, Zhun Liu, Vishrav Chaudhary, Xia Song, Furu Wei
- 2022 (modified: 03 Nov 2022)
- CoRR 2022
- Readers: Everyone
XLM-E: Cross-lingual Language Model Pre-training via ELECTRA
Zewen Chi, Shaohan Huang, Li Dong, Shuming Ma, Bo Zheng, Saksham Singhal, Payal Bajaj, Xia Song, Xian-Ling Mao, Heyan Huang, Furu Wei
- 2022 (modified: 08 Jan 2023)
- ACL (1) 2022
- Readers: Everyone
On the Representation Collapse of Sparse Mixture of Experts
Zewen Chi, Li Dong, Shaohan Huang, Damai Dai, Shuming Ma, Barun Patra, Saksham Singhal, Payal Bajaj, Xia Song, Furu Wei
- 2022 (modified: 08 Jan 2023)
- CoRR 2022
- Readers: Everyone

View all 27 publications

Co-Authors

View all 79 co-authors