ICLR 2025 Workshop SCOPE Submissions

Inference Optimal VLMs Need Fewer Visual Tokens and More Parameters
Kevin Li, Sachin Goyal, João D. Semedo, J Zico Kolter
- Published: 05 Mar 2025, Last Modified: 14 Apr 2025
- SCOPE - ICLR 2025 Poster
- Readers: Everyone
Yes, Q-learning Helps Offline In-Context RL
Denis Tarasov, Alexander Nikulin, Ilya Zisman, Albina Klepach, Andrei Polubarov, Lyubaykin Nikita, Alexander Derevyagin, Igor Kiselev, Vladislav Kurenkov
- Published: 05 Mar 2025, Last Modified: 14 Apr 2025
- SCOPE - ICLR 2025 Poster
- Readers: Everyone
FedEx-LoRA: Exact Aggregation for Federated and Efficient Fine-Tuning of Foundation Models
Raghav Singhal, Kaustubh Ponkshe, Praneeth Vepakomma
- Published: 05 Mar 2025, Last Modified: 14 Apr 2025
- SCOPE - ICLR 2025 Poster
- Readers: Everyone
ChameleonLLM: Batch-Aware Dynamic Low-Rank Adaptation via Inference-Time Clusters
Kamer Ali Yuksel, Hassan Sawaf
- Published: 05 Mar 2025, Last Modified: 14 Apr 2025
- SCOPE - ICLR 2025 Poster
- Readers: Everyone
A Unified Approach to Routing and Cascading for LLMs
Jasper Dekoninck, Maximilian Baader, Martin Vechev
- Published: 05 Mar 2025, Last Modified: 14 Apr 2025
- SCOPE - ICLR 2025 Poster
- Readers: Everyone
Conformal Transformations for Symmetric Power Transformers
Saurabh Kumar, Jacob Buckman, Carles Gelada, Xiaowen Zhang
- Published: 05 Mar 2025, Last Modified: 14 Apr 2025
- SCOPE - ICLR 2025 Poster
- Readers: Everyone
Training Domain Draft Models for Speculative Decoding: Best Practices and Insights
Fenglu Hong, Ravi Shanker Raju, Jonathan Lingjie Li, Bo Li, Urmish Thakker, Avinash Ravichandran, Swayambhoo Jain, Changran Hu
- Published: 05 Mar 2025, Last Modified: 14 Apr 2025
- SCOPE - ICLR 2025 Poster
- Readers: Everyone
Grams: Gradient Descent with Adaptive Momentum Scaling
Yang Cao, Xiaoyu Li, Zhao Song
- Published: 05 Mar 2025, Last Modified: 14 Apr 2025
- SCOPE - ICLR 2025 Poster
- Readers: Everyone
Fast Gradient Computation for RoPE Attention in Almost Linear Time
Yifang Chen, Jiayan Huo, Xiaoyu Li, Yingyu Liang, Zhenmei Shi, Zhao Song
- Published: 05 Mar 2025, Last Modified: 14 Apr 2025
- SCOPE - ICLR 2025 Poster
- Readers: Everyone
Domain-Invariant Prompt Learning for Vision-Language Models
Arsham Gholamzadeh Khoee, Yinan Yu, Robert Feldt
- Published: 05 Mar 2025, Last Modified: 14 Apr 2025
- SCOPE - ICLR 2025 Poster
- Readers: Everyone
Low-Rank Continual Personalization of Diffusion Models
Łukasz Staniszewski, Katarzyna Zaleska, Kamil Deja
- Published: 05 Mar 2025, Last Modified: 14 Apr 2025
- SCOPE - ICLR 2025 Poster
- Readers: Everyone
Efficient Distributed Optimization under Heavy-Tailed Noise
Su Hyeong Lee, Manzil Zaheer, Tian Li
- Published: 05 Mar 2025, Last Modified: 14 Apr 2025
- SCOPE - ICLR 2025 Poster
- Readers: Everyone
Enhanced Continual Learning of Vision-Language Models with Model Fusion
Haoyuan Gao, Zicong Zhang, Yuqi Wei, Linglan Zhao, Guilin Li, Yexin Li, Linghe Kong, Weiran Huang
- Published: 05 Mar 2025, Last Modified: 14 Apr 2025
- SCOPE - ICLR 2025 Poster
- Readers: Everyone
RecurFormer: Not All Transformer Heads Need Self-Attention
RuiqingYan, Linghan Zheng, Xingbo Du, Han Zou, Yufeng Guo, Jianfei Yang
- Published: 05 Mar 2025, Last Modified: 14 Apr 2025
- SCOPE - ICLR 2025 Poster
- Readers: Everyone
XAMBA: Enabling Efficient State Space Models on Resource-Constrained Neural Processing Units
Arghadip Das, Arnab Raha, Shamik Kundu, Soumendu Kumar Ghosh, Deepak Mathaikutty, Vijay Raghunathan
- Published: 05 Mar 2025, Last Modified: 14 Apr 2025
- SCOPE - ICLR 2025 Poster
- Readers: Everyone
SageAttention2: Efficient Attention with Smoothing Q and Per-thread Quantization
Jintao Zhang, Haofeng Huang, Pengle Zhang, Jia wei, Jun Zhu, Jianfei Chen
- Published: 05 Mar 2025, Last Modified: 14 Apr 2025
- SCOPE - ICLR 2025 Oral
- Readers: Everyone
DARS : ROBUST SPARSE FINE-TUNING WITH REGULARIZED SUBSPACE DISALIGNMENT
Sumin Park, Noseong Park
- Published: 05 Mar 2025, Last Modified: 14 Apr 2025
- SCOPE - ICLR 2025 Poster
- Readers: Everyone
Towards Infinite-Long Prefix in Transformers
Yingyu Liang, Zhenmei Shi, Zhao Song, Chiwun Yang
- Published: 05 Mar 2025, Last Modified: 14 Apr 2025
- SCOPE - ICLR 2025 Oral
- Readers: Everyone
In-batch Ensemble Drafting: Robust Speculative Decoding for LVLMs
Minjae Lee, Wonjun Kang, Byeongkeun Ahn, Christian Classen, Minghao Yan, Hyung Il Koo, Kangwook Lee
- Published: 05 Mar 2025, Last Modified: 14 Apr 2025
- SCOPE - ICLR 2025 Poster
- Readers: Everyone
OPPA: OPtimizing PArallelism for Language Model Training
Apivich Hemachandra, Yizhan Han, See-Kiong Ng, Bryan Kian Hsiang Low
- Published: 05 Mar 2025, Last Modified: 14 Apr 2025
- SCOPE - ICLR 2025 Poster
- Readers: Everyone
N-Gram Induction Heads for In-Context RL: Improving Stability and Reducing Data Needs
Ilya Zisman, Alexander Nikulin, Viacheslav Sinii, Denis Tarasov, Lyubaykin Nikita, Andrei Polubarov, Igor Kiselev, Vladislav Kurenkov
- Published: 05 Mar 2025, Last Modified: 14 Apr 2025
- SCOPE - ICLR 2025 Poster
- Readers: Everyone
Initialization using Update Approximation is a Silver Bullet for Extremely Efficient Low-Rank Fine-Tuning
Kaustubh Ponkshe, Raghav Singhal, Eduard Gorbunov, Alexey Tumanov, Samuel Horváth, Praneeth Vepakomma
- Published: 05 Mar 2025, Last Modified: 14 Apr 2025
- SCOPE - ICLR 2025 Poster
- Readers: Everyone
M2R2: EFFICIENT TRANSFORMERS WITH MIXTURE OF MULTI-RATE RESIDUALS
Nikhil Bhendawade, Mahyar Najibi, Devang Naik, Irina Belousova
- Published: 05 Mar 2025, Last Modified: 14 Apr 2025
- SCOPE - ICLR 2025 Oral
- Readers: Everyone
Mixture-of-Mamba: Enhancing Multi-Modal State-Space Models with Modality-Aware Sparsity
Weixin Liang, Junhong Shen, Genghan Zhang, Ning Dong, Luke Zettlemoyer, LILI YU
- Published: 05 Mar 2025, Last Modified: 14 Apr 2025
- SCOPE - ICLR 2025 Oral
- Readers: Everyone
AdaPTS: Adapting Univariate Foundation Models to Probabilistic Multivariate Time Series Forecasting
Abdelhakim Benechehab, Vasilii Feofanov, Giuseppe Paolo, Albert Thomas, Maurizio Filippone, Balázs Kégl
- Published: 05 Mar 2025, Last Modified: 14 Apr 2025
- SCOPE - ICLR 2025 Poster
- Readers: Everyone