ICML 2025 Workshop ES-FoMo-III Submissions

Efficient-vDiT: Efficient Video Diffusion Transformers With Attention Tile
Hangliang Ding, Dacheng Li, Runlong Su, Peiyuan Zhang, Zhijie Deng, Ion Stoica, Hao Zhang
- Published: 11 Jun 2025, Last Modified: 10 Jul 2025
- ES-FoMo III
- Readers: Everyone
Exchangeability in Neural Network Architectures and its Application to Dynamic Pruning
Pu Luke Yi, Tianlang Chen, Yifan Yang, Sara Achour
- Published: 11 Jun 2025, Last Modified: 10 Jul 2025
- ES-FoMo III
- Readers: Everyone
Scaling Fine-Grained MoE Beyond 50B Parameters: Empirical Evaluation and Practical Insights
Jakub Krajewski, Marcin Chochowski, Daniel Korzekwa
- Published: 11 Jun 2025, Last Modified: 10 Jul 2025
- ES-FoMo III
- Readers: Everyone
Outlier-Free Genomic Foundation Models for Resource-Efficient Training and Low-Bit Inference
Chenghao Qiu, Haozheng Luo, Maojiang Su, Zhihan Zhou, Zoe Mehta, Guo Ye, Jerry Yao-Chieh Hu, Han Liu
- Published: 11 Jun 2025, Last Modified: 10 Jul 2025
- ES-FoMo III
- Readers: Everyone
Tensor Product Attention Is All You Need
Yifan Zhang, Yifeng Liu, Huizhuo Yuan, Zhen Qin, Yang Yuan, Quanquan Gu, Andrew C Yao
- Published: 11 Jun 2025, Last Modified: 10 Jul 2025
- ES-FoMo III
- Readers: Everyone
Balancing LoRA Performance and Efficiency with Simple Shard Sharing
Jiale Kang, Qingyu Yin
- Published: 11 Jun 2025, Last Modified: 10 Jul 2025
- ES-FoMo III
- Readers: Everyone
Adaptive Self-improvement LLM Agentic System for ML Library Development
Genghan Zhang, Weixin Liang, Olivia Hsu, Kunle Olukotun
- Published: 11 Jun 2025, Last Modified: 10 Jul 2025
- ES-FoMo III
- Readers: Everyone
Predictive Scheduling for Efficient Inference-Time Reasoning in Large Language Models
Aneesh Muppidi, Katrina Brown, Rana Shahout
- Published: 11 Jun 2025, Last Modified: 10 Jul 2025
- ES-FoMo III
- Readers: Everyone
LongSpec: Long-Context Lossless Speculative Decoding with Efficient Drafting and Verification
Penghui Yang, Cunxiao Du, Fengzhuo Zhang, Haonan Wang, Tianyu Pang, Chao Du, Bo An
- Published: 11 Jun 2025, Last Modified: 10 Jul 2025
- ES-FoMo III
- Readers: Everyone
Shrinking the Generation-Verification Gap with Weak Verifiers
Jon Saad-Falcon, E. Kelly Buchanan, Mayee F Chen, Tzu-Heng Huang, Brendan McLaughlin, Tanvir Bhathal, Shang Zhu, Ben Athiwaratkun, Frederic Sala, Scott Linderman, Azalia Mirhoseini, Christopher Re
- Published: 11 Jun 2025, Last Modified: 10 Jul 2025
- ES-FoMo III
- Readers: Everyone
Multi-student Diffusion Distillation for Better One-step Generators
Yanke Song, Jonathan Lorraine, Weili Nie, Karsten Kreis, James Lucas
- Published: 11 Jun 2025, Last Modified: 10 Jul 2025
- ES-FoMo III
- Readers: Everyone
Zero-Shot Conversion to Monarch-Structured Attention
Can Yaras, Alec S. Xu, Pierre Abillama, Changwoo Lee, Laura Balzano
- Published: 11 Jun 2025, Last Modified: 10 Jul 2025
- ES-FoMo III
- Readers: Everyone
ThinkingViT: Nested Thinking Vision Transformer for Elastic Inference
Ali Hojjat, Janek Haberer, Soren Pirk, Olaf Landsiedel
- Published: 11 Jun 2025, Last Modified: 10 Jul 2025
- ES-FoMo III
- Readers: Everyone
Mu-Parametrization for Mixture of Experts
Jan Małaśnicki, Kamil Ciebiera, Mateusz Boruń, Maciej Pióro, Jan Ludziejewski, Maciej Stefaniak, Michał Krutul, Sebastian Jaszczur, Marek Cygan, Kamil Adamczewski, Jakub Krajewski
- Published: 11 Jun 2025, Last Modified: 10 Jul 2025
- ES-FoMo III
- Readers: Everyone
A Survey on Prompt Tuning
Zongqian Li, Yixuan Su, Nigel Collier
- Published: 11 Jun 2025, Last Modified: 10 Jul 2025
- ES-FoMo III
- Readers: Everyone
PT-MoE: An Efficient Finetuning Framework for Integrating Mixture-of-Experts into Prompt Tuning
Zongqian Li, Yixuan Su, Nigel Collier
- Published: 11 Jun 2025, Last Modified: 10 Jul 2025
- ES-FoMo III
- Readers: Everyone
Ultra-Efficient and Effective Large Language Models with Multi-Boolean Architectures
Ba-Hien Tran, Van Minh Nguyen
- Published: 11 Jun 2025, Last Modified: 10 Jul 2025
- ES-FoMo III
- Readers: Everyone
Batch-Max: Higher LLM Throughput using Larger Batch Sizes and KV Cache Compression
Michael R. Metel, Boxing Chen, Mehdi Rezagholizadeh
- Published: 11 Jun 2025, Last Modified: 10 Jul 2025
- ES-FoMo III
- Readers: Everyone
MuLoCo: Muon is a practical inner optimizer for DiLoCo
Benjamin Thérien, Xiaolong Huang, Irina Rish, Eugene Belilovsky
- Published: 11 Jun 2025, Last Modified: 10 Jul 2025
- ES-FoMo III
- Readers: Everyone
Radio: Rate–Distortion Optimization for Large Language Model Compression
Sean I. Young
- Published: 11 Jun 2025, Last Modified: 10 Jul 2025
- ES-FoMo III
- Readers: Everyone
JSONSchemaBench: Evaluating Constrained Decoding with LLMs on Efficiency, Coverage and Quality
Saibo Geng, Hudson Cooper, Michal Moskal, Samuel Jenkins, Julian Berman, Nathan Ranchin, Robert West, Eric Horvitz, Harsha Nori
- Published: 11 Jun 2025, Last Modified: 10 Jul 2025
- ES-FoMo III
- Readers: Everyone