Mostofa Patwary

Principal Researcher, NVIDIA

Joined

November 2021

Names

Mostofa Patwary (Preferred)

Emails

****@nvidia.com (Confirmed)

Personal Links

Career & Education History

Principal Researcher

NVIDIA (nvidia.com)

2019 – Present

Researcher

Baidu (baidu.com)

2017 – 2019

Researcher

Intel (intel.com)

2013 – 2017

Advisors, Relations & Conflicts

Postdoc Advisor

Alok Choudhary

2012 – 2014

PhD Advisor

Fredrik Manne

2008 – 2012

Expertise

Natural Language Processing

Large Scale Deep Learning

and High Performance Computing.

2011 – Present

Publications

Front-Loading Reasoning: The Synergy between Pretraining and Post-Training Data
Syeda Nahida Akter, Shrimai Prabhumoye, Eric Nyberg, Mostofa Patwary, Mohammad Shoeybi, Yejin Choi, Bryan Catanzaro
- ICLR 2026 Poster
- Readers: Everyone
FusionFactory: Fusing LLM Capabilities with Multi-LLM Log Data
Tao Feng, Haozhen Zhang, Zijie Lei, Pengrui Han, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro, Jiaxuan You
- ICLR 2026 Conference Withdrawn Submission
- Readers: Everyone
Multi-Agent Evolve: LLM Self-Improve through Multi-Agent Co-evolution
Yixing Chen, Yiding Wang, Haofei Yu, Tao Feng, Siqi Zhu, Muhan Zhang, Mostofa Patwary, Jiaxuan You
- Submitted to ICLR 2026
- Readers: Everyone
Nemotron-CC-Math: A 133 Billion-Token-Scale High Quality Math Pretraining Dataset
Rabeeh Karimi mahabadi, Sanjeev Satheesh, Shrimai Prabhumoye, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro
- ICLR 2026 Poster
- Readers: Everyone
RLP: Reinforcement as a Pretraining Objective
Ali Hatamizadeh, Syeda Nahida Akter, Shrimai Prabhumoye, Jan Kautz, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro, Yejin Choi
- ICLR 2026 Poster
- Readers: Everyone
Llama-Nemotron: Efficient Reasoning Models
Soumye Singhal, Jiaqi Zeng, Alexander Bukharin, Yian Zhang, Gerald Shen, Ameya Sunil Mahabaleshwarkar, Bilal Kartal, Yoshi Suhara, Akhiad Bercovich, Itay Levy, Izik Golan, Mohammed Dabbah, Ran El-Yaniv, Somshubra Majumdar, Igor Gitman, Evelina Bakhturina, Jimmy J. Zhang, Bor-Yiing Su, Guyue Huang, Izzy Putterman et al. (7 additional authors not shown)
- EXAIT@ICML 2025 Poster
- Readers: Everyone
Prismatic Synthesis: Gradient-based Data Diversification Boosts Generalization in LLM Reasoning
Jaehun Jung, Seungju Han, Ximing Lu, Skyler Hallinan, David Acuna, Shrimai Prabhumoye, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro, Yejin Choi
- NeurIPS 2025 spotlight
- Readers: Everyone
Efficient Hybrid Language Model Compression through Group-Aware SSM Pruning
Ali Taghibakhshi, Sharath Turuvekere Sreenivas, Saurav Muralidharan, Marcin Chochowski, Yashaswi Karnati, Raviraj Bhuminand Joshi, Ameya Sunil Mahabaleshwarkar, ZIJIA CHEN, Yoshi Suhara, Oluwatobi Olabiyi, Daniel Korzekwa, Mostofa Patwary, Mohammad Shoeybi, Jan Kautz, Bryan Catanzaro, Ashwath Aithal, Nima Tajbakhsh, Pavlo Molchanov
- NeurIPS 2025 poster
- Readers: Everyone
Nemotron-CLIMB: Clustering-based Iterative Data Mixture Bootstrapping for Language Model Pre-training
Shizhe Diao, Yu Yang, Yonggan Fu, Xin Dong, Dan SU, Markus Kliegl, ZIJIA CHEN, Peter Belcak, Yoshi Suhara, Hongxu Yin, Mostofa Patwary, Yingyan Celine Lin, Jan Kautz, Pavlo Molchanov
- NeurIPS 2025 Datasets and Benchmarks Track spotlight
- Readers: Everyone
MIND: Math Informed syNthetic Dialogues for Pretraining LLMs
Syeda Nahida Akter, Shrimai Prabhumoye, John Kamalu, Sanjeev Satheesh, Eric Nyberg, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro
- ICLR 2025 Poster
- Readers: Everyone

View all 45 publications

Co-Authors

View all 143 co-authors