ICML 2025 Workshop ES-FoMo-III Submissions

Learning to Discover Abstractions for LLM Reasoning
Yuxiao Qu, Anikait Singh, Yoonho Lee, Amrith Setlur, Ruslan Salakhutdinov, Chelsea Finn, Aviral Kumar
- Published: 11 Jun 2025, Last Modified: 10 Jul 2025
- ES-FoMo III Spotlight
- Readers: Everyone
SARA: Selective and Adaptive Retrieval-augmented Generation with Context Compression
Yiqiao Jin, Kartik Sharma, Vineeth Rakesh, Yingtong Dou, Menghai Pan, Mahashweta Das, Srijan Kumar
- Published: 11 Jun 2025, Last Modified: 10 Jul 2025
- ES-FoMo III
- Readers: Everyone
Efficient Pre-Training of LLMs via Topology-Aware Communication Alignment on More Than 9600 GPUs
Guoliang He, Youhe Jiang, Wencong Xiao, Jiang Kaihua, Shuguang Wang, Jun Wang, Du Zixian, Zhuo Jiang, Xinlei Zhang, Binhang Yuan, Eiko Yoneki
- Published: 11 Jun 2025, Last Modified: 10 Jul 2025
- ES-FoMo III
- Readers: Everyone
Steering LLM Reasoning Through Bias-Only Adaptation
Viacheslav Sinii, Alexey Gorbatovski, Artem Cherepanov, Boris Shaposhnikov, Nikita Balagansky, Daniil Gavrilov
- Published: 11 Jun 2025, Last Modified: 10 Jul 2025
- ES-FoMo III
- Readers: Everyone
WINA: Weight Informed Neuron Activation for Accelerating Large Language Model Inference
Sihan Chen, Dan Zhao, Jongwoo Ko, Colby Banbury, Huiping Zhuang, Luming Liang, Tianyi Chen
- Published: 11 Jun 2025, Last Modified: 10 Jul 2025
- ES-FoMo III
- Readers: Everyone
Optimal Formats for Weight Quantisation
Douglas Orr, Luka Ribar, Carlo Luschi
- Published: 11 Jun 2025, Last Modified: 10 Jul 2025
- ES-FoMo III
- Readers: Everyone
VOCABTRIM: Vocabulary Pruning for Efficient Speculative Decoding in LLMs
Raghavv Goel, Sudhanshu Agrawal, Mukul Gagrani, Junyoung Park, Yifan Zao, He Zhang, Tian Liu, Yiping Yang, Xin Yuan, Jiuyuan Lu, Christopher Lott, Mingu Lee
- Published: 11 Jun 2025, Last Modified: 10 Jul 2025
- ES-FoMo III
- Readers: Everyone
Q-Adam-mini: Memory-Efficient 8-bit Quantized Optimizer for Large Language Model Training
Yizhou Han, Chaohao Yang, Congliang Chen, Xingjian Wang, Ruoyu Sun
- Published: 11 Jun 2025, Last Modified: 10 Jul 2025
- ES-FoMo III
- Readers: Everyone
Early Attentive Sparsification Accelerates Neural Speech Transcription
Zifei Xu, Sayeh Sharify, Hesham Mostafa, Tristan J Webb, Wanzin Yazar, Xin Wang
- Published: 11 Jun 2025, Last Modified: 10 Jul 2025
- ES-FoMo III
- Readers: Everyone
Vision Language Model Distillation Using Partial Information Decomposition
Stephen D. Liang
- Published: 11 Jun 2025, Last Modified: 10 Jul 2025
- ES-FoMo III
- Readers: Everyone
Making Small Language Models Efficient Reasoners: Intervention, Supervision, Reinforcement
Xuechen Zhang, Zijian Huang, Chenshun Ni, Ziyang Xiong, Jiasi Chen, Samet Oymak
- Published: 11 Jun 2025, Last Modified: 10 Jul 2025
- ES-FoMo III
- Readers: Everyone
BREAD: Branched Rollouts from Expert Anchors Bridge SFT & RL for Reasoning
Xuechen Zhang, Zijian Huang, Yingcong Li, Chenshun Ni, Jiasi Chen, Samet Oymak
- Published: 11 Jun 2025, Last Modified: 10 Jul 2025
- ES-FoMo III
- Readers: Everyone
Fed-SB: A Silver Bullet for Extreme Communication Efficiency and Performance in (Private) Federated LoRA Fine-Tuning
Raghav Singhal, Kaustubh Ponkshe, Rohit Vartak, Lav R. Varshney, Praneeth Vepakomma
- Published: 11 Jun 2025, Last Modified: 10 Jul 2025
- ES-FoMo III
- Readers: Everyone
PoLAR: Polar-Decomposed Low-Rank Adapter Representation
Kai Lion, Liang Zhang, Bingcong Li, Niao He
- Published: 11 Jun 2025, Last Modified: 10 Jul 2025
- ES-FoMo III
- Readers: Everyone
Unbounded Memory and Consistent Imagination via Unified Diffusion–SSM World Models
Jia-Hua Lee, Bor-Jiun Lin, Wei-Fang Sun, Chun-Yi Lee
- Published: 11 Jun 2025, Last Modified: 10 Jul 2025
- ES-FoMo III
- Readers: Everyone
LOGAH: Initialize Large Transformers via Small Graph HyperNetworks
Xinyu Zhou, Boris Knyazev, Alexia Jolicoeur-Martineau, Jie Fu
- Published: 11 Jun 2025, Last Modified: 10 Jul 2025
- ES-FoMo III
- Readers: Everyone
SortedRL: Accelerating RL Training for LLMs through Online Length-aware Scheduling
Yiqi Zhang, Huiqiang Jiang, Xufang Luo, Zhihe Yang, Chengruidong Zhang, Yifei Shen, Dongsheng Li, Yuqing Yang, Lili Qiu, Yang You
- Published: 11 Jun 2025, Last Modified: 10 Jul 2025
- ES-FoMo III
- Readers: Everyone
DEL-ToM: Inference-Time Scaling for Theory-of-Mind Reasoning via Dynamic Epistemic Logic
Yuheng Wu, Jianwen Xie, Denghui Zhang, Zhaozhuo Xu
- Published: 11 Jun 2025, Last Modified: 10 Jul 2025
- ES-FoMo III
- Readers: Everyone
Next-Token Prediction Should be Ambiguity-Sensitive : A Meta-Learing Perspective
Leo Gagnon, Eric Elmoznino, Sarthak Mittal, Tom Marty, Tejas Kasetty, Dhanya Sridhar, Guillaume Lajoie
- Published: 11 Jun 2025, Last Modified: 10 Jul 2025
- ES-FoMo III
- Readers: Everyone
Partition Generative Modeling: Masked Modeling Without Masks
Justin Deschenaux, Lan Tran, Caglar Gulcehre
- Published: 11 Jun 2025, Last Modified: 10 Jul 2025
- ES-FoMo III
- Readers: Everyone
Compressing Large Language Models to Any Size Without Re-Computation
Martin Genzel, Patrick Putzky, Pengfei Zhao, Sebastian Schulze, Mattes Mollenhauer, Robert Seidel, Stefan Dietzel, Thomas Wollmann
- Published: 11 Jun 2025, Last Modified: 10 Jul 2025
- ES-FoMo III
- Readers: Everyone
ABBA: Highly Expressive Hadamard Product Adaptation for Large Language Models
Raghav Singhal, Kaustubh Ponkshe, Rohit Vartak, Praneeth Vepakomma
- Published: 11 Jun 2025, Last Modified: 10 Jul 2025
- ES-FoMo III Spotlight
- Readers: Everyone
LoRA Merging with SVD: Understanding Interference and Preserving Performance
Dennis Tang, Prateek Yadav, Yi-Lin Sung, Jaehong Yoon, Mohit Bansal
- Published: 11 Jun 2025, Last Modified: 10 Jul 2025
- ES-FoMo III
- Readers: Everyone
Towards Understanding Orthogonalization in Muon
Valentyn Boreiko, Zhiqi Bu, Sheng Zha
- Published: 11 Jun 2025, Last Modified: 10 Jul 2025
- ES-FoMo III
- Readers: Everyone
$\mu$-MoE: Test-Time Pruning as Micro-Grained Mixture-of-Experts
Toshiaki Koike-Akino, Jing Liu, Ye Wang
- Published: 11 Jun 2025, Last Modified: 10 Jul 2025
- ES-FoMo III
- Readers: Everyone