ICLR 2025 Workshop MCDC Submissions

ReMod: Learning Structured Sparsity with ReLU Modulation
Wenbo Zhang, Xiang Ren
- Published: 06 Mar 2025, Last Modified: 05 Apr 2025
- MCDC @ ICLR 2025
- Readers: Everyone
How to Merge Multimodal Models Over Time?
Sebastian Dziadzio, Vishaal Udandarao, Karsten Roth, Ameya Prabhu, Zeynep Akata, Samuel Albanie, Matthias Bethge
- Published: 06 Mar 2025, Last Modified: 05 Apr 2025
- MCDC @ ICLR 2025
- Readers: Everyone
A Framework for Double-Blind Federated Adaptation of Foundation Models
Nurbek Tastan, Karthik Nandakumar
- Published: 06 Mar 2025, Last Modified: 04 Apr 2025
- MCDC @ ICLR 2025
- Readers: Everyone
Disentangling Sequence Memorization and General Capability in Large Language Models
Gaurav Rohit Ghosal, Pratyush Maini, Aditi Raghunathan
- Published: 06 Mar 2025, Last Modified: 06 Mar 2025
- MCDC @ ICLR 2025
- Readers: Everyone
Exploring Sparse Adapters for Scalable Merging of Parameter Efficient Experts
Samin Yeasar Arnob, Zhan Su, Minseon Kim, Oleksiy Ostapenko, Doina Precup, Lucas Caccia, Alessandro Sordoni
- Published: 06 Mar 2025, Last Modified: 08 Apr 2025
- MCDC @ ICLR 2025
- Readers: Everyone
Training Plug n' Play Knowledge Modules with Deep Context Distillation
Lucas Caccia, Alan Ansell, Ivan Vulić, Edoardo Ponti, Alessandro Sordoni
- Published: 06 Mar 2025, Last Modified: 06 Mar 2025
- MCDC @ ICLR 2025
- Readers: Everyone
FedMoDN: Federated Modular Decision Support Networks
Cécile Trottet, Michael Krauthammer, Mary-Anne Hartley
- Published: 06 Mar 2025, Last Modified: 05 Apr 2025
- MCDC @ ICLR 2025
- Readers: Everyone
Soup-of-Experts: Pretraining Specialist Models via Parameters Averaging
Pierre Ablin, Angelos Katharopoulos, Skyler Seto, David Grangier
- Published: 06 Mar 2025, Last Modified: 06 Mar 2025
- MCDC @ ICLR 2025
- Readers: Everyone
On-Device Collaborative Language Modeling via a Mixture of Generalists and Specialists
Dongyang Fan, Bettina Messmer, Nikita Doikov, Martin Jaggi
- Published: 06 Mar 2025, Last Modified: 06 Mar 2025
- MCDC @ ICLR 2025
- Readers: Everyone
Improving the Efficiency of Distributed Training using Sparse Parameter Averaging
Matt Beton, Matthew Reed, Seth Howes, Alex Cheema, Mohamed Baioumy
- Published: 06 Mar 2025, Last Modified: 07 Apr 2025
- MCDC @ ICLR 2025
- Readers: Everyone
An Empirical Study of Policy Interpolation via Diffusion Models
Yuqing Xie, Chao Yu, Ya Zhang, Yu Wang
- Published: 06 Mar 2025, Last Modified: 06 Apr 2025
- MCDC @ ICLR 2025
- Readers: Everyone
Tight Clusters Make Specialized Experts
Stefan Nielsen, Rachel S.Y. Teo, Laziz Abdullaev, Tan Minh Nguyen
- Published: 06 Mar 2025, Last Modified: 06 Mar 2025
- MCDC @ ICLR 2025
- Readers: Everyone
Exact Unlearning of Finetuning Data via Model Merging at Scale
Kevin Kuo, Amrith Setlur, Kartik Srinivas, Aditi Raghunathan, Virginia Smith
- Published: 06 Mar 2025, Last Modified: 06 Mar 2025
- MCDC @ ICLR 2025
- Readers: Everyone
Multi-Agent Verification: Scaling Test-Time Compute with Multiple Verifiers (Abridged)
Shalev Lifshitz, Sheila A. McIlraith, Yilun Du
- Published: 06 Mar 2025, Last Modified: 22 Mar 2025
- MCDC @ ICLR 2025
- Readers: Everyone
Mastering Massive Multi-Task Reinforcement Learning via Mixture-of-Expert Decision Transformer
Yilun Kong, Guozheng Ma, Qi Zhao, Haoyu Wang, Li Shen, Xueqian Wang, Dacheng Tao
- Published: 06 Mar 2025, Last Modified: 25 Mar 2025
- MCDC @ ICLR 2025
- Readers: Everyone
Truncate without Fear: Module Aggregation and Redistribution in Federated Low-Rank Adaptation
Zhijie Chen, Yuxing Liu, Arindam Banerjee
- Published: 06 Mar 2025, Last Modified: 05 Apr 2025
- MCDC @ ICLR 2025
- Readers: Everyone
Exploring Asynchronism in SWARM Parallelism
Yan Zuo, Gil Avraham, Thalaiyasingam Ajanthan, Sameera Ramasinghe, Alexander Long
- Published: 06 Mar 2025, Last Modified: 02 Apr 2025
- MCDC @ ICLR 2025
- Readers: Everyone
Beyond Top-K: Structured Sparsification for Compression in Pipeline Parallel
Sameera Ramasinghe, Thalaiyasingam Ajanthan, Gil Avraham, Yan Zuo, Alexander Long
- Published: 06 Mar 2025, Last Modified: 04 Apr 2025
- MCDC @ ICLR 2025
- Readers: Everyone
MoLEx: Mixture of Layer Experts for Finetuning with Sparse Upcycling
Rachel S.Y. Teo, Tan Minh Nguyen
- Published: 06 Mar 2025, Last Modified: 06 Mar 2025
- MCDC @ ICLR 2025
- Readers: Everyone
ROBUST ONLINE INFERENCE USING ADAPTIVE MODEL SWITCHING
Kalpan Mukherjee, Vikramank Singh, Abishek Sankararaman, Balakrishnan Murali Narayanaswamy, Tim Kraska
- Published: 06 Mar 2025, Last Modified: 18 Mar 2025
- MCDC @ ICLR 2025
- Readers: Everyone
HDEE: Heterogeneous Domain Expert Ensemble
Oguzhan Ersoy, Jari Kolehmainen, Gabriel Passamani Andrade
- Published: 06 Mar 2025, Last Modified: 04 Apr 2025
- MCDC @ ICLR 2025
- Readers: Everyone
Momentum Look-Ahead for Asynchronous Distributed Low-Communication Training
Thalaiyasingam Ajanthan, Sameera Ramasinghe, Gil Avraham, Yan Zuo, Alexander Long
- Published: 06 Mar 2025, Last Modified: 31 Mar 2025
- MCDC @ ICLR 2025
- Readers: Everyone
Adaptive Local Training in Federated Learning
Donald Shenaj, Eugene Belilovsky, Pietro Zanuttigh
- Published: 06 Mar 2025, Last Modified: 04 Apr 2025
- MCDC @ ICLR 2025
- Readers: Everyone
Revisiting Sparse Mixture of Experts for Resource-adaptive Federated Fine-tuning Foundation Models
Van-Tuan Tran, Le Huy Khiem, Quoc-Viet Pham
- Published: 06 Mar 2025, Last Modified: 04 Apr 2025
- MCDC @ ICLR 2025
- Readers: Everyone
Conditioning on Local Statistics for Scalable Heterogeneous Federated Learning (Tiny Paper)
Rickard Brannvall
- Published: 06 Mar 2025, Last Modified: 31 Mar 2025
- MCDC @ ICLR 2025
- Readers: Everyone