Zain Sarwar

Joined

September 2023

Names

Zain Sarwar (Preferred)

Emails

****@uchicago.edu (Confirmed)

Personal Links

Career & Education History

PhD student

University of Chicago (uchicago.edu)

2021 – 2025

Advisors, Relations & Conflicts

PhD Advisor

2023 – 2026

Expertise

Language modeling

2021 – Present

Deep Learning

2020 – Present

Publications

Structured Thoughts For Improved Reasoning And Context Pruning
Zain Sarwar, Supriyo Chakraborty, Berkcan Kapusuzoglu, Chia-Hsuan Lee, Anirban Das, Stephen Rawls, Kartik Balasubramaniam, Sambit Sahu
- ICLR 2026 Conference Withdrawn Submission
- Readers: Everyone
Low Rank Experts Enable Specialization In Dense Transformers
Zain Sarwar, Ashwinee Panda, Benjamin Thérien, Milind Naphade, Sambit Sahu, Supriyo Chakraborty
- ICLR 2026 Conference Withdrawn Submission
- Readers: Everyone
Continual Pre-training of MoEs: How robust is your router?
Benjamin Thérien, Charles-Étienne Joseph, Zain Sarwar, Ashwinee Panda, Anirban Das, Shi-Xiong Zhang, Stephen Rawls, Sambit Sahu, Eugene Belilovsky, Irina Rish
- CCFM Poster
- Readers: Everyone
Continual Pre-training of MoEs: How robust is your router?
Benjamin Thérien, Charles-Étienne Joseph, Zain Sarwar, Ashwinee Panda, Anirban Das, Shi-Xiong Zhang, Stephen Rawls, Sambit Sahu, Eugene Belilovsky, Irina Rish
- Accepted by TMLR
- Readers: Everyone
Dense Backpropagation Improves Training for Sparse Mixture-of-Experts
Ashwinee Panda, Vatsal Baherwani, Zain Sarwar, Benjamin Thérien, Sambit Sahu, Tom Goldstein, Supriyo Chakraborty
- NeurIPS 2025 poster
- Readers: Everyone
Dense Backpropagation Improves Routing for Sparsely-Gated Mixture-of-Experts
Ashwinee Panda, Vatsal Baherwani, Zain Sarwar, Benjamin Thérien, Stephen Rawls, Sambit Sahu, Supriyo Chakraborty, Tom Goldstein
- Compression Workshop @ NeurIPS 2024
- Readers: Everyone
Dense Backpropagation Improves Routing for Sparsely-Gated Mixture-of-Experts
Ashwinee Panda, Vatsal Baherwani, Zain Sarwar, Benjamin Thérien, Stephen Rawls, Sambit Sahu, Supriyo Chakraborty, Tom Goldstein
- NeurIPS 2024 Workshop
- Readers: Everyone
Mycroft: Towards Effective and Efficient External Data Augmentation
Zain Sarwar, Van Tran, Arjun Nitin Bhagoji, Nick Feamster, Ben Y. Zhao, Supriyo Chakraborty
- Submitted to ICLR 2025
- Readers: Everyone
StructMoE: Augmenting MoEs with Hierarchically Routed Low Rank Experts
Zain Sarwar, Ashwinee Panda, Benjamin Thérien, Stephen Rawls, Sambit Sahu, Supriyo Chakraborty
- ICLR 2025 Conference Withdrawn Submission
- Readers: Everyone
Dense Backpropagation Improves Routing for Sparsely-Gated Mixture-of-Experts
Ashwinee Panda, Vatsal Baherwani, Zain Sarwar, Benjamin Thérien, Stephen Rawls, Sambit Sahu, Tom Goldstein, Supriyo Chakraborty
- Submitted to ICLR 2025
- Readers: Everyone

Co-Authors