ICML 2023 Workshop ES-FoMO Submissions

More Context, Less Distraction: Improving Zero-Shot Inference of CLIP by Inferring and Describing Spurious Features
Bang An, Sicheng Zhu, Michael-Andrei Panaitescu-Liess, Chaithanya Kumar Mummadi, Furong Huang
- Published: 20 Jun 2023, Last Modified: 16 Jul 2023
- ES-FoMO 2023 Poster
- Readers: Everyone
Predictive Pipelined Decoding: A Compute-Latency Trade-off for Exact LLM Decoding
Seongjun Yang, Gibbeum Lee, Jaewoong Cho, Dimitris Papailiopoulos, Kangwook Lee
- Published: 20 Jun 2023, Last Modified: 16 Jul 2023
- ES-FoMO 2023 Poster
- Readers: Everyone
Generating Efficient Kernels for Quantized Inference on Large Language Models
Tommaso Pegolotti, Elias Frantar, Dan Alistarh, Markus Püschel
- Published: 20 Jun 2023, Last Modified: 16 Jul 2023
- ES-FoMO 2023 Poster
- Readers: Everyone
SpeedLimit: Neural Architecture Search for Quantized Transformer Models
Yuji Chai, Luke Bailey, Yunho Jin, Glenn Ko, Matthew Karle, David Brooks, Gu-Yeon Wei, H. Kung
- Published: 20 Jun 2023, Last Modified: 16 Jul 2023
- ES-FoMO 2023 Poster
- Readers: Everyone
A Comprehensive Analysis of Adapter Efficiency
Nandini Mundra, Sumanth Doddapaneni, Raj Dabre, Anoop Kunchukuttan, Ratish Puduppully, Mitesh M Khapra
- Published: 20 Jun 2023, Last Modified: 16 Jul 2023
- ES-FoMO 2023 Poster
- Readers: Everyone
Less is More: Using Multiple LLMs for Applications with Lower Costs
Lingjiao Chen, Matei Zaharia, James Zou
- Published: 20 Jun 2023, Last Modified: 16 Jul 2023
- ES-FoMO 2023 Poster
- Readers: Everyone
Blockwise Parallel Transformer for Long Context Large Models
Hao Liu, Pieter Abbeel
- Published: 20 Jun 2023, Last Modified: 16 Jul 2023
- ES-FoMO 2023 Poster
- Readers: Everyone
SuperShaper: A Pre-Training Approach for Discovering Efficient Transformer Shapes
Vinod Ganesan, Gowtham Ramesh, Pratyush Kumar, Raj Dabre
- Published: 20 Jun 2023, Last Modified: 16 Jul 2023
- ES-FoMO 2023 Poster
- Readers: Everyone
Continual Pre-Training of Large Language Models: How to re-warm your model?
Kshitij Gupta, Benjamin Thérien, Adam Ibrahim, Mats Leon Richter, Quentin Gregory Anthony, Eugene Belilovsky, Irina Rish, Timothée Lesort
- Published: 20 Jun 2023, Last Modified: 16 Jul 2023
- ES-FoMO 2023 Poster
- Readers: Everyone
Implementing block-sparse matrix multiplication kernels using Triton
Priya Mishra, Trevor Gale, Matei Zaharia, Cliff Young, Deepak Narayanan
- Published: 20 Jun 2023, Last Modified: 16 Jul 2023
- ES-FoMO 2023 Poster
- Readers: Everyone
Looped Transformers are Better at Learning Learning Algorithms
Liu Yang, Kangwook Lee, Robert D Nowak, Dimitris Papailiopoulos
- Published: 20 Jun 2023, Last Modified: 16 Jul 2023
- ES-FoMO 2023 Poster
- Readers: Everyone
Accelerating LLM Inference with Staged Speculative Decoding
Benjamin Frederick Spector, Christopher Re
- Published: 20 Jun 2023, Last Modified: 16 Jul 2023
- ES-FoMO 2023 Poster
- Readers: Everyone
Test-Time Training for Speech
Sri Harsha Dumpala, Chandramouli Shama Sastry, Sageev Oore
- Published: 20 Jun 2023, Last Modified: 16 Jul 2023
- ES-FoMO 2023 Poster
- Readers: Everyone
Towards Efficient World Models
Eloi Alonso, Vincent Micheli, François Fleuret
- Published: 20 Jun 2023, Last Modified: 16 Jul 2023
- ES-FoMO 2023 Poster
- Readers: Everyone
The Framework Tax: Disparities Between Inference Efficiency in Research and Deployment
Jared Fernandez, Jacob Kahn, Clara Na, Yonatan Bisk, Emma Strubell
- Published: 20 Jun 2023, Last Modified: 16 Jul 2023
- ES-FoMO 2023 Poster
- Readers: Everyone
Towards Structured Sparsity in Transformers for Efficient Inference
Harry Dong, Beidi Chen, Yuejie Chi
- Published: 20 Jun 2023, Last Modified: 16 Jul 2023
- ES-FoMO 2023 Poster
- Readers: Everyone
H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models
Zhenyu Zhang, Ying Sheng, Tianyi Zhou, Tianlong Chen, Lianmin Zheng, Ruisi Cai, Zhao Song, Yuandong Tian, Christopher Re, Clark Barrett, Zhangyang Wang, Beidi Chen
- Published: 20 Jun 2023, Last Modified: 16 Jul 2023
- ES-FoMO 2023 Poster
- Readers: Everyone
Incrementally-Computable Neural Networks: Efficient Inference for Dynamic Inputs
Or Sharir, Anima Anandkumar
- Published: 20 Jun 2023, Last Modified: 16 Jul 2023
- ES-FoMO 2023 Poster
- Readers: Everyone
Compositional Interfaces for Compositional Generalization
Jelena Luketina, Jack Lanchantin, Sainbayar Sukhbaatar, Arthur Szlam
- Published: 20 Jun 2023, Last Modified: 16 Jul 2023
- ES-FoMO 2023 Poster
- Readers: Everyone
ZipLM: Inference-Aware Structured Pruning of Language Models
Eldar Kurtic, Elias Frantar, Dan Alistarh
- Published: 20 Jun 2023, Last Modified: 16 Jul 2023
- ES-FoMO 2023 Poster
- Readers: Everyone
Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training
Hong Liu, Zhiyuan Li, David Leo Wright Hall, Percy Liang, Tengyu Ma
- Published: 20 Jun 2023, Last Modified: 16 Jul 2023
- ES-FoMO 2023 Poster
- Readers: Everyone
MosaicBERT: How to Train BERT with a Lunch Money Budget
Jacob Portes, Alexander R Trott, Sam Havens, DANIEL KING, Abhinav Venigalla, Moin Nadeem, Nikhil Sardana, Daya Khudia, Jonathan Frankle
- Published: 20 Jun 2023, Last Modified: 16 Jul 2023
- ES-FoMO 2023 Poster
- Readers: Everyone
UOTA: Unsupervised Open-Set Task Adaptation Using a Vision-Language Foundation Model
Youngjo Min, Kwangrok Ryoo, Bumsoo Kim, Taesup Kim
- Published: 20 Jun 2023, Last Modified: 16 Jul 2023
- ES-FoMO 2023 Poster
- Readers: Everyone
Cramming: Training a Language Model on a single GPU in one day
Jonas Geiping, Tom Goldstein
- Published: 20 Jun 2023, Last Modified: 16 Jul 2023
- ES-FoMO 2023 Poster
- Readers: Everyone
SpecTr: Fast Speculative Decoding via Optimal Transport
Ziteng Sun, Ananda Theertha Suresh, Jae Hun Ro, Ahmad Beirami, Himanshu Jain, Felix Yu, Michael Riley, Sanjiv Kumar
- Published: 20 Jun 2023, Last Modified: 16 Jul 2023
- ES-FoMO 2023 Poster
- Readers: Everyone